Supporting Research on Today’s Optical Network Infrastructure Presented by Jerry Keith Director of Communications University of California, Riverside UCCS Aug. 2, 2004
The Story of the Trailer • The Call • The Visits • Another Call • Visit Again • New Information • What We Did
The Call • One hot customer – had talked to a number of people, not satisfied • Well punctuated conversation • By the end of the conversation I knew exactly how important this project was
The Visits • Visit from desktop staff • Visit from network staff • Actual hubs, put together locally • They asserted it had been working fine • Theory of virus attack • There really was one, we cured it • We relaxed – problem solved
Another Call • Different PI • More facts, specific scenarios • The person made sense • The virus had not been the problem • It was time for another visit
Visit Again • Asked staff to return and look again • We sent a multidiscipline team • Looked farther afield • Found additional facts
New Information • Local copper T1 on less than Cat 3 • Shared a small router with all other T1’s • Interference from large, multi-day transfer into off-campus site • Researchers had moved their two projects voluntarily to facilitate function in their home department
What We Did & Are Doing • Installed a Wireless hop and new switch gear to improve the local situation • Moving the off campus facility with heavy traffic to dark fiber we are leasing from the City of Riverside • Move this project’s servers to an on-campus site with good connectivity
The Story of the Trailer - Findings • Wrong place • Temporary facilities—no fiber planned • Nearly impossible to upgrade the copper path • Wrong technology • T1 not Ethernet • Antiquated router, OK for low use T1s • Wrong network assumptions • PIs assumed they could move anywhere on campus and have service • Wrong support structure • Sent one discipline at a time • No formal mechanism to get wider set of expertise on the problem quickly • Wrong planning • Could say, they should have asked us • Our business is research, their processing must work • We believe that we must be proactive in this arena
The Story of the Trailer • All was well until the Research units moved in: • E-mail worked • Web surfing worked • Administrative applications also
Research & Systems • The relationship between research and systems is different than a few years ago: • more emphasis on computing and communications, joint or matrixed computing efforts are common • This essential because it would take lifetimes to analyze some data sets without improved technology
Faster More powerful Grid computing at many levels E.g. Teragrid Research & Systems Computing
Faster More Capacity Relatively Cheaper Transmission alternatives – WDM Can be structured for specific purposes Research & Systems Networks
The CalREN Network CENIC Experimental Network CalREN XP CENIC High Performance Research Network CalREN HPR CENIC Digital California Network CalREN DC The CalREN network has three distinct service structures to support day to day requirements as well as research needs.
CalREN Structure • The CalREN backbone is based on dark fiber optic lines throughout the state • Each network runs on its own wavelength or color of light • The technology is called DWDM or Dense Wave Division Multiplexing • It is very cost effective to expand this type of network either for more speed or to add additional networks. • The next slide depicts this schematically ----
CalREN DC/HPR/NLR PoP Architecture CalREN DC & HPR Existing 7500 Experimental 10Gig E Gig E Campus or Metro Interconnect One of above options used for Each campus CalREN DC HPR NLR Long Haul OC48/OC192/10GigE DWDM DWDM 10 Gig E or OC192 Gig E CalREN/DC HPR NLR 15500
CalREN BackboneTopology Seattle Portland Chico Nat. Lambda Rail 10G Tier 2 10G Tier 3 2.5G Metro ring dark fiber (Tiers 2 and 3) Emeryville SF Sacramento PAIX-Palo Alto Sunnyvale UCSC Stockton/ Modesto Salinas UCM Fresno CPSLO San Luis Obispo A Statewide View of the CalREN Backbone Bakersfield UCSB Santa Barbara Los Angeles Anaheim WestEd Santa Ana San Diego SDSC 8 Nov 2001
Campus Connections • The next slide shows the campus connections • Each Campus has two separate connections to CalREN HPR and to CalREN DC • Each campus can reflect this structure onto their campus net as they see fit. -these slides courtesy of Gregg Scott at CENIC
Default Last Mile Design Specification ONI Demarc Gig Enet Hub Site “A” Tier 2 CENIC CENIC Tier 3 Campus Network Campus/Institution Last Miles Hub Site “B” Tier 2 CENIC CENIC Tier 3 Gig Enet
CalREN Networks CENIC Experimental Network CalREN XP CENIC High Performance Research Network CalREN HPR CENIC Digital California Network CalREN DC CalREN DC provides internet connectivity for typical networking needs, CalREN HPR is focused on providing high performance for research while the XP network opens the door to experimenting with networks or very unique connectivity for projects.
Research • Impact of Research on Networks and Computing • Moving data • Processing Data • Testing protocols and transmission • Not every researcher understands the recent improvements in networks or how to use them.
Interesting sites • TeraGrid • http://www.teragrid.org/ • National LambdaRail [NLR] • http://www.nlr.net/
A Quick Sidebar on UCR Fiber • UCR uses air-blown fiber [ABF] from Sumitomo • Install tubing, then blow fiber strands with compressed air • Possible to pull out and replace with ease • Easy to add new runs by blowing new strands • Used by Getty Museum, Department of Defense, and others
Another Researcher Calls • Slow data transfers • We thought we knew the solution • New fiber to building (ABF) • New Switch for building entry • New gigabit connection to campus core • This appears to be the solution • But, you don’t always know
Who knows what’s out there? • Firewalls, slow networks, broken protocols, old servers, etc. • To support research you need to know
End to End Performance Testing • Interenet2 has End to End Performance Initiative • Look on www.internet2.edu • Click on link “E2EPI” • They initially thought this would be a short stop-gap • There is lots to read about here
E2EPI Goals • Enable end-users and support personnel to: • Determine E2E performance capabilities • Locate E2E problems • Contact the right person to get an E2E problem resolved • Enable remote initiation of partial path performance tests • Make partial path performance data publicly available
Support of local users and researchers • Good documentation • Team approach – multi discipline • Communicate effectively with end users • Develop relationship with VC for Research • Standards for networking assumptions made in grant requests • Maintain E2E perspective
Research Opportunities Facilitated by CalREN HPR and XP Architecture An Opportunity to think about Research and Extramurally Funded Opportunities in a New Way • Creating Private, Ad Hoc Networks • Data Collection Devices, Servers, Data, and Databases can be linked across campus, the UC system, or even throughout the United States via ultra-high speed networks. UC’s ability to create ad hoc, private networks (on-campus and within the UC system) allows researchers to engage in unique inter- and intra-campus collaborations, to share equipment and systems, and to address problems and propose solutions utilizing innovative methodologies. Ultimately, UC’s network infrastructure provides researchers with an important tool to help secure research (and other extramurally funded) opportunities.
One Research Opportunity Facilitated by UCR’s Network Infrastructure UCR’s Infomine Project, A Campus Example • This extramurally funded project will build “virtual library collections” for campus faculty through focused web “crawling” utilizing very high amounts of bandwidth. • How does UCR’s Network Facilitate this Effort? • Air Blown Fiber. • Electronics Supporting Direct Connection to Internet 2. • Network Traffic Shaping to Hold Down Costs. • This project will consume double the existing amount of campus network bandwidth WITHOUT IMPACTING THE “PRODUCTION NETWORK” AND WITHOUT DRAMATICALLY INCREASED INTERNET COSTS.
Logical Overview of “Private Network” ProvidingBandwidth to Focused Web Crawling Initiative
Research Opportunities Facilitated via UCR’s Network Infrastructure California UCR CENIC Experimental Network Private Experimental Network CENIC High Performance Research Network Private and High Availability Research Network CENIC Digital California Network High Availability Academic and Administrative Network • Why Are Private Research Networks Important? • Dedicated Bandwidth • Not Subject to “Normal” Campus Rules • Can be Constructed to Individual Project or Researcher’s Needs
Next Steps for UCR… Cross-functional team to support this effort. • Network, Systems, and Security. • Grant Writing Expertise. • Collaboratory Support for Research Design and Methodology Support. • Announcement in the Fall 2004.
For further consideration … The Oceanstore Project http://oceanstore.cs.berkeley.edu/info/overview.html Using Networks to promote Collaborative Research http://www-itg.lbl.gov/~johnston/Virtual.Labs.html The Archipelago Project http://www.cnds.jhu.edu/research/networks/archipelago/ The Archipelago Project investigates efficient ways to form a secure extended ad-hoc network of laptops, handhelds, and other wireless capable devices, and bridge it to the Internet. • Interesting Projects * Comments and Questions are welcome: jerry.keith @ucr.edu