1.29k likes | 1.56k Views
Introduction to High-Performance Internet Switches and Routers. Network Architecture. Long Haul Network. DWDM. Core. 10 GbE. Core. Core Routers. 10 GbE. Core Routers. Metropolitan. Metropolitan. Campus / Residential. 10 GbE. Edge switch. Edge Routers. 10 GbE. GbE. • • •.
E N D
Introduction to High-Performance Internet Switches and Routers
Network Architecture Long Haul Network DWDM Core 10GbE Core Core Routers 10GbE Core Routers Metropolitan Metropolitan Campus / Residential 10GbE Edge switch Edge Routers 10GbE GbE • • • • • • Access switch Access Routers http://www.ust.hk/itsc/network/
pop pop pop pop
How the Internet really is: Current Trend Modems, DSL SONET/SDH DWDM
What is Routing? R3 R1 R4 D A B E R2 C R5 F 5
Points of Presence (POPs) POP3 POP2 POP1 D POP4 A B E POP5 POP6 C POP7 POP8 F 6
Where High Performance Routers are Used (10 Gb/s) R2 (10 Gb/s) R1 R6 R5 R4 R7 R3 R9 R10 R8 R11 R12 R14 R13 R16 R15 (10 Gb/s) (10 Gb/s) 049045 - Router Architectures 7
Hierarchical arrangement End hosts (1000s per mux) Access multiplexer Edge Routers POP Core Routers 10Gb/s “OC192” POP POP POP: Point of Presence. Richly interconnected by mesh of long-haul links. Typically: 40 POPs per national network operator; 10-40 core routers per POP. Point of Presence (POP)
Typical POP Configuration Transport Network DWDM/SONET Terminal Backbone routers 10G WAN Transport Links > 50% of high speed interfaces are router-to-router (Core routers) 10G Router-Router Intra-Office Links Aggregation switches/routers (Edge Switches)
Today’s Network Equipment Routers Switches SONET DWDM LAYER 3 LAYER 2 LAYER 1 LAYER 0 Internet FR & ATM SONET DWDM Protocol
Interconnect Egress linecard Ingress linecard Route lookup Buffer ing Framing TTL process ing Framing Buffer ing QoS schedul ing Interconnect scheduling Control plane Control path Data path Scheduling path Functions in a packet switch
Ingress linecard Egress linecard Interconnect Framing Framing Interconnect scheduling Control plane Control path Data path Functions in a circuit switch
Our emphasis for now is to look at packet switches (IP, ATM, Ethernet, Framerelay, etc.)
What a Router Looks Like Cisco CRS-1 Juniper T1600 60 cm 44 cm Capacity: 640Gb/sPower: 13.2kWFull rack Capacity: 1.6Tb/sPower: 9.1kWHalf-a-rack 214 cm 95 cm 101cm 79 cm (16-Slot Single-Shelf System) (16-Slot System) 049045 - Router Architectures 14
What a Router Looks Like Cisco GSR 12416 Juniper M160 19” 19” Capacity: 160Gb/sPower: 4.2kW Capacity: 80Gb/sPower: 2.6kW 6ft 3ft 2ft 2.5ft
A Router Chassis Fans/ Power Supplies Linecards
A Circuit Board with connectors for line cards High speed electrical traces connecting line cards to fabric Usually passive Typically 30-layer boards Backplane
Cisco Catalyst 3750G Cisco CRS-1 What do these two have in common?
CRS-1 linecard 20” x (18”+11”) x 1RU 40Gbps, 80MPPS State-of-the-art 0.13u silicon Full IP routing stack including IPv4 and IPv6 support Distributed IOS Multi-chassis support Cat 3750G Switch 19” x 16” x 1RU 52Gpbs, 78 MPPS State-of-the-art 0.13u silicon Full IP routing stack including IPv4 and IPv6 support Distributed IOS Multi-chassis support What do these two have in common?
Cisco Catalyst 3750G Cisco CRS-1 What is different between them?
CRS-1 linecard Up to 1024 linecards Fully programmable forwarding 2M prefix entries and 512K ACLs 46Tbps 3-stage switching fabric MPLS support H-A non-stop routing protocols Cat 3750G Switch Up to 9 stack members Hardwired ASIC forwarding 11K prefix entries and 1.5K ACLs 32Gbps sharedstack ring L2 switching support Re-startable routing applications A lot…
Other packet switches Cisco 7500 “edge” routers Lucent GX550 Core ATM switch DSL router
D D R3 R1 R4 D A B E R2 C R5 Destination Next Hop F D R3 E R3 F R5 What is Routing?
R3 R1 R4 D D A D D 1 4 16 32 D Ver HLen T.Service Total Packet Length Fragment ID Flags Fragment Offset B E TTL Protocol Header Checksum 20 bytes Source Address R2 C R5 Destination Address Destination Next Hop F D R3 Options (if any) E R3 Data F R5 What is Routing?
R3 R1 R4 D A B E R2 C R5 F What is Routing?
Basic Architectural Elementsof a Router • Routing • Routing table update • (OSPF, RIP, IS-IS) • Admission Control • Congestion Control • Reservation Control Plane “Typically in Software” • Routing • Lookup • Packet • Classifier • Switching • Arbitration • Scheduling Switch (per-packet processing) “Typically in Hardware” Switching
Basic Architectural ComponentsDatapath: per-packet processing 3. 1. Output Scheduling 2. Forwarding Table Interconnect Forwarding Decision Forwarding Table Forwarding Decision Forwarding Table Forwarding Decision
Per-packet processing in a Switch/Router 1. Accept packet arriving on an ingress line. 2. Lookup packet destination address in the forwarding table, to identify outgoing interface(s). 3. Manipulate packet header: e.g., decrement TTL, update header checksum. 4. Send packet to outgoing interface(s). 5. Queue until line is free. 6. Transmit packet onto outgoing line.
ATM Switch • Lookup cell VCI/VPI in VC table. • Replace old VCI/VPI with new. • Forward cell to outgoing interface. • Transmit cell onto link.
Ethernet Switch • Lookup frame DA in forwarding table. • If known, forward to correct port. • If unknown, broadcast to all ports. • Learn SA of incoming frame. • Forward frame to outgoing interface. • Transmit frame onto link.
IP Router • Lookup packet DA in forwarding table. • If known, forward to correct port. • If unknown, drop packet. • Decrement TTL, update header Cksum. • Forward packet to outgoing interface. • Transmit packet onto link.
Special per packet/flow processing • The router can be equipped with additional capabilities to provide special services on a per-packet or per-class basis. • The router can perform some additional processing on the incoming packets: • Classifying the packet • IPv4, IPv6, MPLS, ... • Delivering packets according to a pre-agreed service: Absolute service or relative service (e.g., send a packet within a given deadline, give a packet a better service than another packet (IntServ – DiffServ)) • Filtering packets for security reasons • Treating multicast packets differently from unicast packets
Packet processing must be simple and easy to implement • Memory access time is the bottleneck • 200Mpps × 2 lookups/pkt = 400 Mlookups/sec → 2.5ns per lookup Per packet Processing Must be Fast !!!
CPU Buffer Memory Route Table CPU Line Interface Line Interface Line Interface Memory MAC MAC MAC Typically <0.5Gb/s aggregate capacity First Generation Routers Shared Backplane Line Interface
Bus-based Router Architectures with Single Processor • The first generation of IP router • Based on software implementations on a single general-purpose CPU. • Limitations: • Serious processing bottleneck in the central processor • Memory intensive operations (e.g. table lookup & data movements) limits the effectiveness of processor power • A severe limiting factor to overall router throughput from input/output (I/O) bus
Fwding Cache Second Generation Routers CPU Buffer Memory Route Table Line Card Line Card Line Card Buffer Memory Buffer Memory Buffer Memory Fwding Cache Fwding Cache MAC MAC MAC Typically <5Gb/s aggregate capacity
Bus-based Router Architectures with Multiple Processors • Architectures with Route Caching • Distribute packet forwarding operations • Network interface cards • Processors • Route caches • Packets are transmitted once over the shared bus • Limitations: • The central routing table is a bottleneck at high-speeds • traffic dependent throughput (cache) • shared bus is still a bottleneck
Fwding Table Third Generation Routers Switched Backplane Line Card CPU Card Line Card Local Buffer Memory Local Buffer Memory Line Interface CPU Routing Table Memory Fwding Table MAC MAC Typically <50Gb/s aggregate capacity
Switch-based Router Architectures with Fully Distributed Processors • To avoid bottlenecks: • Processing power • Memory bandwidth • Internal bus bandwidth • Each network interface is equipped with appropriate processing power and buffer space.
Fourth Generation Routers/SwitchesOptics inside a router for the first time Optical links 100s of metres Switch Core Linecards 0.3 - 10Tb/s routers in development
Juniper TX8/T640 Alcatel 7670 RSP Avici TSR Chiaro
Next Gen. Backbone Network Architecture – One backbone, multiple access networks Dual Stack IPv4-IPv6 Cable Network CE router (G)MPLS based Multi-service Intelligent Packet Backbone Network Dual Stack IPv4-IPv6 Enterprise Network PE PE Router (Service POP) Service POP Residential PE DSL, FTTH, Dial CE router GGSN SGSN CE router ISP’s IPv6 IX Telecommuter Dual Stack IPv4-IPv6 DSL/FTTH/Dial access Network ISP offering Native IPv6 services • One Backbone Network • Maximizes speed, flexibility and manageability
Data Hdr Data Hdr IP Address Next Hop Address Table Buffer Memory ~1M prefixes Off-chip DRAM ~1M packets Off-chip DRAM Current Generation: Generic Router Architecture Header Processing Lookup IP Address Update Header Queue Packet
Data Data Data Data Data Data Hdr Hdr Hdr Hdr Hdr Hdr Header Processing Header Processing Header Processing Lookup IP Address Lookup IP Address Lookup IP Address Update Header Update Header Update Header 1 1 Address Table Address Table Address Table 2 2 N N Current Generation: Generic Router Architecture (IQ) Queue Packet Buffer Memory Queue Packet Buffer Memory Scheduler Queue Packet Buffer Memory
Data Data Data Hdr Hdr Hdr Header Processing Header Processing Header Processing Lookup IP Address Lookup IP Address Lookup IP Address Update Header Update Header Update Header Address Table Address Table Address Table Current Generation: Generic Router Architecture (OQ) 1 1 Queue Packet Buffer Memory 2 2 Queue Packet Buffer Memory N N Queue Packet Buffer Memory
Basic Architectural Elementsof a Current Router Typical IP Router Linecard Buffer & State Memory Scheduler Buffer Mgmt & Scheduling Physical Layer Framing & Maintenance Buffered or Bufferless Fabric (e.g. crossbar, bus) Packet Processing Buffer Mgmt & Scheduling Lookup Tables Buffer & State Memory OC192c Linecard: ~10-30M gates ~2Gbits of memory ~2 square feet >$10k cost; price $100K • Backplane
WFQ Performance metrics • Capacity • “maximize C, s.t. volume < 2m3 and power < 5kW” • Throughput • Operators like to maximize usage of expensive long-haul links. • Controllable Delay • Some users would like predictable delay. • This is feasible with output-queueing plus weighted fair queueing (WFQ).
Why do we Need Faster Routers? • To prevent routers from becoming the bottleneck in the Internet. • To increase POP capacity, and to reduce cost, size and power.
Why we Need Faster RoutersTo prevent routers from being the bottleneck Line Capacity 2x / 7 months User Traffic 2x / 12months Router Capacity 2.2x / 18months Moore’s Law 2x / 18 months DRAM Random Access Time 1.1x / 18months