Internet POPs, Telecom Hotels, and Internet Data Centers

1. Internet POPs, Telecom Hotels, and Internet Data Centers CS 294-3 – The Converged Network Spring 2002 George Porter U.C. Berkeley -- EECS

2. Internet: collection of networks • This talk is about connectivity and computation • How do the networks in the Internet communicate with each other? • What do transit providers do with their traffic? • Motivation for computation in the network (Internet Data Centers) U.C. Berkeley -- EECS

3. Computers increasingly connected U.C. Berkeley -- EECS

4. U.C. Berkeley -- EECS

5. U.C. Berkeley -- EECS

6. NSFnet • 1987-1995 • Managed by Merit • ANS, IBM, MCI, State of Michigan • Consisted of T-1 connections • In 1992, moved to T-3 links run by Advanced Network & Services (ANSnet) U.C. Berkeley -- EECS

7. Post NFSnet • vBNS (very high-speed backbone network services) run by MCI • Additional NAPs (Network Access Points) • MAE-East, D.C. – MFS Datanet (now MCI Worldcom) • Ameritech, Chicago – Ameritech • PacBell, San Jose – PacBell • Sprint, Pennsauken, NJ - Sprint U.C. Berkeley -- EECS

8. ATLnap (Atlanta) Bellcore Multimedia exchange NY6iX – New York IPv6 MAE-LA Seattle IX MAE-Houston PAIX Equinix eXchange Linx (London) FreeIX (France) AMS-IX (Amsterdam) etc Many new Commercial NAPs U.C. Berkeley -- EECS

9. Internet POPs: Two Examples • NeoSoft Inc., Houston Texas • AMS-IX (Amsterdam) U.C. Berkeley -- EECS

10. Internet POPs: Two Examples • NeoSoft Inc., Houston Texas • AMS-IX (Amsterdam) U.C. Berkeley -- EECS

11. BBS in 1989, became an ISP in 1992 • Founder Karl Lehenbauer (Left) • I worked there from 1994-1999 U.C. Berkeley -- EECS

12. Initial Connectivity • Originally, NeoSoft connected to NSFnet via Sesquinet @ Rice • T1 cost $14k up front, $2k/month • Cisco router IGS $10k • 198.??.xx.yy from Sesquinet’s address space U.C. Berkeley -- EECS

13. New Connectivity • Eventually bought fractional DS3 from MCI (Sprint wouldn’t route less than /20 due to problems) • On own CIDR block (128 class C’s: 206.109) • BGP-4 running on Cisco 7513 Router • $15k/month U.C. Berkeley -- EECS

14. Peering • Peered with UUnet for 2 months • NeoSoft had a large webserver, and served out much more traffic than inbound • Became member of MAE/Houston • MAGE (Metro Area Gigabit Ethernet) via Phonoscope ($4k/month!) U.C. Berkeley -- EECS

15. Experiences in Peering • Many other local providers didn’t have a clue! • INSYNC misconfigured their routes, and traffic would be outbound through MAGE, inbound through random other routes • Often advertised incorrect BGP updates • PSInet: Advertised Dial-up ISDN U.C. Berkeley -- EECS

16. Hops, Multihoming • In 1994, the Sprint and MCI handoff was in Chicago • 20 hops to next door neighbor • Eventually exchanged in Dallas • Experimented with Multihoming (DS3 and T1 to Cable & wireless) • Not too good, BGP administration • Plan was for recovery U.C. Berkeley -- EECS

17. NeoSoft  Internet America • In 1999, Internet America buys NeoSoft for $8M • Houston office closes • I get laid off  U.C. Berkeley -- EECS

18. Internet POPs: Two Examples • NeoSoft Inc., Houston Texas • AMS-IX (Amsterdam) U.C. Berkeley -- EECS

19. AMS-IX • Carrier-neutral peering point • Founded in 1994 • 100+ members • 565 Tbytes/month • AT&T, Akamai, Dynegy, Digital Island, Deutsche Telecom, France Telecom, Global Crossing, UUnet NL, etc U.C. Berkeley -- EECS

20. Organization U.C. Berkeley -- EECS

21. U.C. Berkeley -- EECS

22. U.C. Berkeley -- EECS

23. How to Join • Companies apply for membership after agreeing to numerous policies (sometimes voting is involved) • Cost • 10baseT = 500,00 euro/month • 100baseT = 850,00 euro/month • 1000base T = 1200,00 euro/month • 1 euro = 0.87 USD (12/26/2002) U.C. Berkeley -- EECS

24. Computation in the network • It makes sense to move servers and content to places of high connectivity • By the economics of scale it is cheaper to provide connectivity, power, management, etc to many customers at a central site U.C. Berkeley -- EECS

25. Example: eXchange • Paul Ave location in SF: 350K sq. ft • People can rent cabinets, racks, cages • In addition to being a data center, also a huge connectivity point U.C. Berkeley -- EECS

26. Services • UPS + generator • Office space • Climate Control • “Meet me” room with carriers and other service providers • Professional monitoring • Security (hand scanners, guards) • Fire control U.C. Berkeley -- EECS

27. waveExchange • Unique facility located across the street from eXchange • 20+ carriers • Near fiber routes/loops • Carriers can meet in “meet me rooms” • Service between ASPs, ISPs, CDNs… • Opened Nov 2, 2001 U.C. Berkeley -- EECS

28. Tenants of waveExchange • AT&T, Cogent, Enron, Level3, PacBell, Qwest, Sigma, Worldcom, Williams Communications, Xo, PAIX, others • Network effect • Huge connectivity + Highly available services U.C. Berkeley -- EECS

29. Summary • Progression from leased lines to NAPs to peering points gives ISPs options • I haven’t even mentioned private peering arrangements • Putting computation in the network at the points of connectivity enables new services that can meet demand U.C. Berkeley -- EECS