1 / 63

L. E. Price Argonne National Laboratory HEP-CCC Meeting CERN, November 10, 2000

ESnet International and Interregional Networking (Original meeting slides can be viewed at http://ccwww.kek.jp/ESI2000/). L. E. Price Argonne National Laboratory HEP-CCC Meeting CERN, November 10, 2000. Agenda: First Day. Monday 24 July

stasia
Download Presentation

L. E. Price Argonne National Laboratory HEP-CCC Meeting CERN, November 10, 2000

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ESnet International and Interregional Networking(Original meeting slides can be viewed at http://ccwww.kek.jp/ESI2000/) L. E. Price Argonne National Laboratory HEP-CCC Meeting CERN, November 10, 2000

  2. Agenda: First Day • Monday 24 July • --------------0900: Welcome0910: Brief reports and updates (15 min each) DANTE Davies DFN Wilhelm/Ernst INFN/GARR Valente UK Hughes-Jones Canada Karlen1030: Break1100: Brief reports and updates (15 min each) ESnet Leighton DOE Seweryniak CERN Martin DESY Ernst KEK/Japan Yuasa1230 LUNCH1345: Brief reports and updates (15 min each) ITER Casci NSF Strawn SLAC Cottrell1445: Montoring results Cottrell1515: Break1545: Discussion: where are the urgent problems?1730: Adjourn

  3. Agenda: Second Day • Tuesday 25 July---------------0900: International Planning (20 minutes each) STAR-TAP Winkler ESnet Testbed Leighton Internet 2 and int'l collaborations Boyles1030: Break1100: Brief report: NII (formerly NACSIS) Asano1130: Initiatives and applications Data Grids Newman Video conferencing Galvez1230: Lunch1400: Discussion and Matters Arising during the meeting Qwest Overview Richeson WDM Karlen Impact of High End Applications Seweryniak International Connections Newman Network Bandwidth Valente Other Regions Cottrell Authentication and Security Williams1530: Break1600: Continue discussion; Action Items1730: Adjourn

  4. Federico Casci /ITER Heather Boyles /Internet2 David Williams /CERN Jim Leighton /ESnet Lisa Erspamer /ESnet Michael Ernst /DESY Terry Shalk /UCSC Les Cottrell /SLAC Dean Karlen /Carleton Harvey Newman /Caltech George Strawn /NSF George Seweryniak /DOE Steve Jardin /PPPL Dai Davies /Dante Larry Price /ANL Philippe Galvez /Caltech Gregory Denis /Caltech Olivier Martin /CERN Enzo Valente /INFN Richard Hughes-Jones /Manchester Fernando Liello /INFN Linda Winkler /STARTAP Anne Richeson /Qwest Shoichiro Asano /NII Toshiaki Matsuda /JAERI Kenzo Tsuda /NIFS Yoshiyuki Watase /KEK Yukio Karita /KEK Fukuko Yuasa /KEK Soh Suzuki /KEK Teiji Nakamura /KEK Youhei Morita /KEK Esnet InternationalKyoto, July 24-25, 2000Participants

  5. TEN-155 : Year Two Dante • 622 MBps Core in 5 Locations • 622 GBps Access Capability • More Multiple Connections • Subscribed Bandwidth 2.5 GBps

  6. Trans-Atlantic Pricing and Volumes Dante

  7. News on TA Connectivity DFN • DFN TA Capacity Upgrade by mid October ‘99 • from 155 Mbps to 4*155 Mbps (ATM, PoS) • Distributed Connections to BWiN • DFN´s PoP in the US moved to NY/Telehouse • Upstream to UUnet upgraded to 4*155Mbp (PoS) • Abilene in Cooperation with DANTE (45 Mbps) • Connecting us to StarTap

  8. 44 34 Mbps 25 155 Mbps DFN’s Trans Atlantic Connectivity ATM (OC-3) POS (OC-3) ATM (OC-3) ATM (OC-3)

  9. DFN’s Connectivity to Europe & Japan Japan 1(2) ATM VC @ 155 Mbps)

  10. Connectivity to North America DFN • Users are suffering from overloaded links • Load still Asymmetric • Overloaded: US => Germany • Currently No dedicated B/W for DESY Connectivity to Canada • Users are suffering from overloaded TA links • Canadian Traffic routed via Commercial Provider DFN => NY => DANTE => AlterNet => CA*NET though there is a dedicated link CA*NET <=> NY

  11. European Connectivity... more sorrows DFN • DFN <=> TEN-155 IX overloaded • Second OC-3 due since weeks Connectivity to Japan • Major Improvement • Physicists are very happy Avg. Volume UoTokyo => DESY is 2 GB/day @ > 100 KB/s • Managed Bandwidth Service to DESY ?

  12. Connectivity to Russia & FSU DESY • Satellite link DESY <=> Moscow • DESY/HEP traffic only • 2 Mbps (by RSCC) • Funded by DESY (6000 $ / month) • Satellite link DESY <=> FSU • Novosibirsk (~1 Mbps), Armenia, Georgia, ... • NATO’s Infrastructure Grant Group • Direct Video Broadcast (Multi Channel) • Fiber on the Horizon (?)

  13. Present HEP activity INFN/GARR • Usual activity with European labs • CERN (LEP & LHC) • DESY (HERA) • BaBar computing regional center in INFN–Rome (CASPUR computing Center) Technical Development • Remote Control Room and Monitoring • FermiLab (CDF testbed) • In Collaboration with TF-TANT & GEANT • Bandwidth Brokerage • QoS across heterogeneous domains • DiffServ in multi-domain environment

  14. Open Issues (North–America) INFN/GARR • Solve Access problems to US Universities (on Abilene or not) • Routing problems • Last-mile bottlenecks • Connectivity with NASA Centers • Same rank as HEP Centers Open Issues (Far–East) • Improve Asia–Pacific Connectivity • Bandwidth • Delay • Connectivity with new Field-Sites • Tibet (ARGO)

  15. Other Issues INFN/GARR • Other Regions • South–America (AUGER) • “GRID” Impact on: • Security Requirements • Network Requirements (Capacity, QoS, etc.)

  16. SuperJANET Connections in the UK Northern College St Andrews College GSA RSAMD Napier ECA Queen Margaret AbMAN RobertGordon 155 Mbps Cleveland College PoP Site Abertay ClydeNET Glasgow CarlisleCollege EaStMAN 34 Mbps New College Durham BEN MAP Aberdeen St Andrews Stirling GlasgowCaledonia FaTMAN Paisley Strathclyde Heriot Watt UHI Edinburgh Menai Dundee Deeside Northumbria Llandrillo Durham ColegLlysfasi Edinburgh Newcastle UNN Daresbury Keele Central Lancaster Teesside Bangor NorMAN WCoH PO Lab Newcastle Staffordshire Liverpool Hope Bradford Hull NEWI Yale Sunderland Sheffield Huddersfield NNW Southport Halton Edge Hill Gla-BEN Ed-BEN York Lancaster YHMAN U-Net Bolton Inst. Ncl-BEN Scarborough UCL ITE Bangor Sheffield Hallam MCC Leeds Metropolitan LCC Liverpool LJMU Leeds Linc & Humb RNCM Derby Nottingham Manchester Metropolitan MCC-BEN Le-BEN HEAnet Ulster G-MING EMMAN CWC CWC Salford Bishop Grosseteste Leicester UMIST Manchester Leeds Nott-BEN Nott Trent BGS Belfast Be-BEN Newman City CollegeManchester Stephenson UC Northampton UCEB BCFTCS Reading Stranmill Surrey Wolverhampton Rdg-BEN London Royal Holloway St Mary BAS Birmingham Homerton Hinxton Hall UEA Bristol Warwick Bham-BEN MidMAN Westminster Worcester Coll. CWC CWC Cranfield Sandwell Roehampton Powys C-BEN Aberystwyth Westhill Aston Cambridge ITE Monkswood Coventry Kingston Harlech Cf-BEN Anglia Poly QMW MST Radar Essex CoventryTechnical College UEL RCA ULCC-BEN Ceredigion Ex-BEN R-BEN ULCC LBS Cardiff Telehouse Meirion-Dwyfor Newport Bris-BEN Imperial LMAN C&G WCMD Ravensbourne Swansea Institute Glamorgan Bristol Soton-BEN O-BEN Greenwich SWMAN Soton Southbank Falmouth RAL ULCC Goldsmiths Mid KentCollege Carmarthen St Davids Soton City UCL Trinity IOE BWEMAN Dartington Oxford Swansea Exeter Cardiff Institute UWE Birkbeck Afan SOAS Bucks See separate diagram Portsmouth Seeseparate diagram Chichester RVC LSHTM St Mark & John UKERNA BGSExeter SWAN Neath Gorseinon Pembroke ChichesterCollege Bath Sussex LondonGuildhall Merthyr Brookes Bournemouth Aberdare Swansea College Brighton Bath College LondonInstitute HEFCE Plymouth Bridgend Lampeter UK

  17. SuperJANET Connections in London • 4 * 155 Mbit links to SuperJANET • 2 * 155 Mbit transatlantic links • 3rd link in use 18 Jul 00 • 4th planned by End of 00

  18. The Grid UK • Keen Interest in the EU DataGrid (CERN) middleware proposal. • HEP experiments will need world Grid access. • Grid Networking interests include: • High throughput, reliable, very large data transfers • QoS issues – diffserv, RSVP and packet marking • Monitoring, Metrics, and prediction accuracy • Modelling • Physics Applications

  19. Conclusion & Status UK • Network Links to US and Europe are OK (at the moment !) • Peering with Abilene, ESnet and CANARIE OK, interesting effects with CAR – WRED ! • Considerable UK interest in the Internet2 tests • Grid: active in several areas including networks • SuperJANET4 rollout is very promising

  20. … to ESnet (via STARTAP) Canada

  21. … to US Universities Canada

  22. … to UK Canada TA via 10 Mbps PVC to Ten-155

  23. … to Germany Canada TA via UK & Ten-155 priority packets

  24. … to Japan (from Carleton) Canada peering with CA*net2/3

  25. International Connectivity Canada • General improvement in connectivity • Canada-Germany not stable • asymmetric routing needs to be sorted out

  26. Canada CA*net 3 National Optical Internet Consortium Partners: Bell Nexxia Nortel Cisco JDS Uniphase Newbridge CA*net 3 Primary Route CA*net 3 Diverse Route GigaPOP ORAN Deploying a 4 channel CWDM Gigabit Ethernet network – 700 km Deploying a 4 channel Gigabit Ethernet transparent optical DWDM– 1500 km Condo Dark Fiber Networks connecting universities and schools Condo Fiber Network linking all universities and hospital Multiple Customer Owned Dark Fiber Networks connecting universities and schools Netera MRnet SRnet ACORN St. John’s BCnet Calgary Regina Winnipeg Charlottetown RISQ ONet Fredericton Montreal Vancouver 16 channel DWDM -8 wavelengths @OC-192 reserved for CANARIE -8 wavelengths for carrier and other customers Halifax Ottawa Seattle STAR TAP Toronto Los Angeles Chicago New York

  27. ESnet

  28. ESnet

  29. KPNQwest co-location (Chicago) CERN Qwest PoP Qwest-IP CERN (Geneva) T3 CERNH9 KPN- Qwest CERN-USA T3 LS1010 STAR TAP T3 T3 LS1010 T3

  30. Internet Access@CERN CERN Canada ESnet Japan Internet2 Abilene STARTAP vBNS SWITCH MREN SURFNET #1.4 Commodity Internet #1.3 JANET #2.1 #1.2 TEN-155 #2.2 DFN CERN PoP USA #1.1 #1 #2 #1 CERN CIXP #4 Mission oriented #3

  31. Main Internet connections@CERN (2) CERN • US Line consortium (USLIC) • CERN, US/HEP (via Caltech & DoE), Canada/HEP (via Carleton) • IN2P3 (CCPN Lyon). • World Health Organization (WHO). • JAPAN • NACSIS (4Mbps ATM/VP over TEN-155’s Managed Bandwidth Service (MBS)) • Genesis & JEG (Japan - Europe Gamma project) projects. • CIXP (CERN Internet Exchange Point)

  32. WAN vs LAN bandwidth CERN • The common belief that WAN will always be well behind LANs (i.e. 1-10%) is, according to me, plain wrong but admittedly controversial. • WAN technology is well ahead of LAN technology, state of the art is 10Gbps (WAN) against 1Gbps (LAN) • Price is less of an issue as they are falling down very rapidly. • Some people are even advocating that one should now start thinking new applications as if bandwidth was free, which sounds a bit premature to me, at least, in Europe, even though there are large amounts of unused capacity floating around!

  33. Cost Analysis and future perspectives CERN • KPNQwest offered an STM-1 upgrade option from 1/10/00: • The circuit can be delivered as one protected or two unprotected SDH circuits, thus effectively doubling the available bandwidth and also almost halving the cost. • As an additional local loop is necessary. • Unfortunately, dual unprotected SDH circuits are only available on the KPNQwest unfrastructure, i.e. up to New-York. • Significant price/performance improvements achieved: • still paying 400KCHF/Mbps/year 16 months ago (Swisscom/MCI), • then 88KCHF/Mbps/year (C&W) • now 36KCHF/Mbps/year (KPN-Qwest) • expect to pay 8KCHF/Mbps/year, if the dual unprotected STM-1 solution is selected. • The combination of prices going down and a moderate budget increase (20-25%) may make it possible to move towards 4*STM-1 faster than originally planned: • But this only makes sense if there are real prospects to make effective use of the capacity end to end, which is far from being the case today.

  34. Which termination point in the USA? CERN • Both New-York (60 Hudson Street) and Chicago (Qwest PoP) have advantages and disadvantages: • New-York (60 Hudson street): • Pros: • direct peering with Abilene, Canarie & Esnet. • availability of dual unprotected SDH transatlantic circuits • Startap International Transit network. • continued need for extension till STAR TAP in Chicago therefore questionable, hence significant saving in sight. • Cons: • manpower, travel, loss of layer 2 connectivity to the STARTAP • Chicago (Qwest NBC building): • Pros: • minimum changes • direct peering with Fermi Lab, Argonne, MREN, Umich • Peering with commercial ISPs. • Cons: • additional cost

  35. Possible upgrade scenarios CERN • Scenario (1): • upgrade protected 45M circuit to protected STM-1 circuit between CERN and Chicago (Qwest NBC building) or N-Y (60 Hudson street) around 1/12/00. • Scenario (2): • upgrade protected 45M circuit to two unprotected STM-1 circuits between CERN and New-York. • Scenario (3): • same as 1 & 2 but with STM-1 instead of 45M to STAR TAP • Scenario (4): • same as 1, 2 & 3 but without direct circuit to STAR TAP (use STAR TAP ITN instead) • N.B. all scenarios are still under active discussion, hence no migration date yet approved, but an STM-1 circuit is likely to be in place before the end of the year.

  36. Technical Challenges CERN • QoS • Very high speed file transfer

  37. CONCLUSIONS CERN • Multiple circuits from CERN to Tier1 regional centers at up to 2.5 Gbps (i.e. STM-16/OC-192c) will be possible by 2003-2005. • Cost may be problematic (1-3MCHF per circuit). • Very high speed LANs implied. • Gigabit/second file transfer on high bandwidth*delay paths may still be problematic. • The public Internet as well as national research networks are evolving in a way nobody can predict. • This will have a profound impact on the LHC

  38. KEK

  39. BGP Connection Summary KEK

  40. Connection to ASIA HENP KEK • KEK - BINP - MSU • Extend routing to Moscow area in Sep 1999 • 2Mbps fiber between BINP and MSU • Carried traffic to/from ITEP (AS2148) • KEK - Academia Sinica (Taiwan) • TAnet (AS1659) - Academia Sinica - KEK • KEK - IHEP (China) • Upgrade to FR PVC is being studied (cost at Chinese side is a problem)

  41. KEK 24 12 12 12 12 12 1.5 2 2 12 256K 256K 2 12 12 2 12

  42. Summary KEK • Infrastructure updates provide us good connectivity to US universities, Europe laboratories. • Still need upgrade of bandwidth to ASIA HENP • How can we construct secure HEP network?

  43. Internet 2 STARTAP IUCC Abilene APAN CA*net3 vBNS CERN SURFnet ESnet NII MIRnet DFN DREN Singaren NORDUnet TAnet NREN INFN GEMNET NISN DANTE SEATTLE Renater CA*net3 JAnet Abilene CA*net3 SURFnet ESnet NORDUnet Miami Developing International Peering NYC (Telehouse 25 Broadway) Chile China NYC (60 Hudson) Abilene ESnet LA vBNS Abilene Singaren CUDI Argentina Chile Columbia

  44. Internet 2 Internet2 Backbone Networks Donna Cox,Robert Patterson, NCSA

  45. Internet 2 Abilene int’l peering OC12 OC3-12 STAR TAP APAN/TransPAC, Canet, IUCC, NORDUnet, RENATER, REUNA, SURFnet, SingAREN, SINET, TAnet2 (CERnet, HARnet) SEATTLE CAnet, (AARnet) NYC DANTE*, JANET, NORDUnet, SURFnet (CAnet) SUNNYVALE (SINET?) L.A. SingAREN, (SINET?) San Diego (CUDI?) MIAMI (CUDI?, REUNA, RNP2, RETINA) El Paso, TX (CUDI?)

  46. Internet 2 AmPATHFlorida International University & Global Crossing

  47. Performance PingER PingER • Measurements from • 30 monitors in 15 countries • Over 600 remote hosts, over 420 sites • 72 countries • Over 2100 monitor-remote site pairs • Recent monitor additions: ANL, UWisc, NSK, ITEP, RIKEN, KAIST, ILAN, Brazil, Melbourne; working on: Caltech, SDSC • Over 50% of HENP collaborator sites are explicitly monitored as remote sites by PingER project • Atlas (37%), BaBar (68%), Belle (23%), CDF (73%), CMS (31%), D0 (60%), LEP (44%), Zeus (35%), PPDG (100%), RHIC(64%) • Remainder covered by Beacons • Recently extended from 56 to 74 • Now allows selection of packet size, # packets • facilitate tuning for low & high performing links

  48. Performance Beacons & UK seen from ESnet Effect of ACLs Direct peering between JANet and ESnet Sites in UK track one another, so can represent with single site 2 Beacons in UK Indicates common source of congestion Increased capacity by 155 times in 5 years

More Related