1 / 28

A 50-Gb/s IP Router

A 50-Gb/s IP Router. Authors: Craig Partridge et al. IEEE/ACM TON June 1998 Presenter: Srinivas R. Avasarala CS Dept., Purdue University. Why a Gigabit Router?. Transmission link bandwidths are improving at very fast rates Network usage is expanding

vida
Download Presentation

A 50-Gb/s IP Router

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. A 50-Gb/s IP Router Authors: Craig Partridge et al. IEEE/ACM TON June 1998 Presenter: Srinivas R. Avasarala CS Dept., Purdue University

  2. Why a Gigabit Router? • Transmission link bandwidths are improving at very fast rates • Network usage is expanding • Host Adapters, OS, switches and MUX also need to get faster for improved network performance • The goal of the work is to show routers can keep pace with other technologies CS Dept., Purdue University

  3. Goals of a Multi-Gigabit Router (MGR) • Enough internal bandwidth to move packets between its interfaces at gigabit rates • Enough packet processing power to forward several million packets per second (MPPS) • Conformance to protocol standards MGR achieves up to 32 MPPS forwarding rates with 50 Gb/s of full duplex backplane capacity CS Dept., Purdue University

  4. Router Architecture • Multiple line cards, each with one or more network interfaces • Forwarding Engine cards (FEs), to make packet forwarding decisions • High speed switch • Network processor CS Dept., Purdue University

  5. Router Architecture CS Dept., Purdue University

  6. Major Innovations • Each FE has a complete set of the routing tables • A switched fabric is used instead of the traditional shared bus • FEs are on boards distinct from the line cards • Use of an abstract link layer header • Include QoS processing in the router CS Dept., Purdue University

  7. The Forwarding Engine Processor • A 415-MHz DEC Alpha 21164 processor • 64bit 32 register super scalar RISC processor • 2 Integer logic units E0, E1 • 2 Floating point logic units FA, FM • Each cycle schedules one instruction to each logic unit, processing 4 instructions (quad) in a group CS Dept., Purdue University

  8. Forwarding Engine’s Caches • 3 internal caches • First level instruction cache (Icache) 8kB • First-level data cache (Dcache) 8kB • An on-chip secondary cache (Scache) 96kB used as a cache of recent routes. Can store 12000 routes approx. with 64b per route • An external tertiary cache (Bcache) 16 MB • Divided into two 8MB banks • One bank stores entire forwarding table • Other is updated by NW processor via PCI bus CS Dept., Purdue University

  9. Forwarding Engine Hardware • Headers are placed in a request FIFO queue • Alpha reads from queue head, examines header, makes route decision and informs inbound card • Header includes 24/56B of packet + 8B abstract link layer header and alpha reads a min of 32B • Alpha writes out the updated header indicating the outbound interface to use (dispatching info) • Updated header contains the outbound link layer address and a flow id used for packet scheduling • Unique approach to ARP !! CS Dept., Purdue University

  10. Forwarding Engine Software • A few 100 lines of code • 85 instructions in the common case taking a minimum of 42 cycles. • This gives a peak forwarding rate of 9.8MPPS • 415 MHz/42 cycles ~ 9.8MPPS • Fast path of the code is in 3 stages, each with about 20-30 instructions (10-15 cycles) CS Dept., Purdue University

  11. Fast path of the code • Stage 1: • Basic error checking to see if header is from a IP datagram • Confirm packet/header lengths are reasonable • Confirm that IP header has no options • Compute hash offset into route cache and load the route • Start loading of next header CS Dept., Purdue University

  12. Fast path of the code • Stage 2 • Check if cached route matches destination of the datagram • If not then do an extended lookup in the route table in Bcache • Update TTL and CHECKSUM fields • Stage 3 • Put updated ttl, checksum and the route information into IP hdr along with link layer info from the forwarding table CS Dept., Purdue University

  13. An exception !! • IP HDR checksum is not verified but only updated • The incremental update algorithm is safe because if the checksum is bad, it remains bad • Reason: Checksum verification is expensive and is a large penalty to pay for a rare error that can be caught end-to-end • Requires 17 instructions with min of 14 cycles, increasing forwarding time by 21% • IPv6 does not include a header checksum too!! CS Dept., Purdue University

  14. Some datagrams not handled in fast path • Headers whose destination misses in the cache • Headers with errors • Headers with IP options • Datagrams that require fragmentation • Multicast datagrams • Requires multicast routing which is based on source address and inbound link as well • Requires multiple copies of header to be sent to different line cards CS Dept., Purdue University

  15. Instruction set • 27% of them do bit, byte or word manipulation due to extraction of various fields from headers • The above instructions can only be done in E0, resulting in contention (checksum verifying) • Floating point instructions account for 12% but do not have any impact on performance as they only set SNMP values and can be interleaved • There is a minimum of loads(6) and stores(4) CS Dept., Purdue University

  16. Issues in forwarding design • Why not use an ASIC in place of the engine ? • Since IP protocol is stable, why not do it ? • Answer depends on where the router will be deployed: corporate LAN or ISP’s backbone? • How effective is a route cache ? • A full route lookup is 5 times more expensive than a cache hit. So we need modest hit rates. • And modest hit rates seem to be assured because of packet trains CS Dept., Purdue University

  17. Abstract link layer header • Designed to keep the forwarding engine and its code simple CS Dept., Purdue University

  18. The Switched bus • Instead of the conventional shared bus, MGR uses a 15-port point-to-point switch • Limitation of a point-to-point switch is that it does not support one to many transfers • The switch has 2 interfaces to each function card • Data Interface: 75 input, 75 output pins clocked at 51.84 MHz • Allocation Interface: 2 request pins, 2 inhibit pins, 1 input status pin and 1 output status pin clocked at 25.92 MHz CS Dept., Purdue University

  19. Data transfer in the switch • An epoch is 16 ticks of data clock (8 allocation clk) • Up to 15 simultaneous transfers in an epoch • Each transfer is 1024 bits of data + 176 auxiliary bits for parity and control • Aggregate data bandwidth is 49.77 Gb/s. 58.32 Gb/s including the auxiliary bits. 3.3 Gb/s per line card • The 1024 bits are sent in two 64B blocks • Function cards are expected to wait several epochs for another 64B block to fill the transfer CS Dept., Purdue University

  20. Scheduling of the switch • Minimum of 4 epochs to schedule and complete a transfer but scheduling is pipelined. • Epoch1: source card signals that it has data to send to the destination card • Epoch2: switch allocator schedules transfer • Epoch3: source and destination cards are notified and told to configure themselves • Epoch4: transfer takes place • Flow control through inhibit pins CS Dept., Purdue University

  21. The Switch Allocator card • Takes connection requests from function cards • Takes inhibit requests from destination cards • Computes a transfer configuration for each epoch • 15X15 = 225 possible pairings with 15! Patterns • Disadvantages of the simple allocator • Unfair: there is a preference for low-numbered sources • Requires evaluating 225 positions per epoch, which is too fast for an FPGA CS Dept., Purdue University

  22. The Switch Allocator Card CS Dept., Purdue University

  23. The Switch Allocator • Solution to unfairness problem: Random shuffling of sources and destinations • Solution to timing problem: Parallel evaluation of multiple locations • Priority to requests from forwarding engines over line cards to avoid header contention on line cards CS Dept., Purdue University

  24. Line Card Design • A line card in MGR can have up to 16 interfaces on it, all of the same type • Total bandwidth of all interfaces on a card must not exceed 2.5 Gb/s. The difference between 2.5 and 3.3 Gb/s is to allow for transfer of headers to and from forwarding engines • Can support: • 1 OC-48c 2.4 Gb/s SONET interface • 4 OC-12c 622 Mb/s SONET interfaces • 3 HIPPI 800 Mb/s interfaces • 16 100Mb/s Ethernet or FDDI interfaces CS Dept., Purdue University

  25. Line card: Inbound packet processing • Assigns a packet id and breaks data into a chain of 64B • The first page is sent to the FE to get routing info • 2 Complications • Multicasting: FE sends multiple copies of updated first pages for a single packet • ATM: Cells are 53b. So we need SAR. OAM cells between interfaces on the same card must be sent directly in a single page. CS Dept., Purdue University

  26. Line card: Outbound packet processing • Receives pages of a packet from the switch • Assembles them in a list • Creates a packet record pointing to the list • Passes the packet record QoS processor (an FPGA) which does scheduling based on flow ids • Any link layer scheduling is done separately later CS Dept., Purdue University

  27. Network processor, Routing tables • 233-MHz 21064 Alpha processor • Access to line cards through a PCI bridge • Runs 1.1 NetBSD UNIX • All routing protocols run on the NW processor • FEs have only small tables with minimal info • NW processor periodically downloads new tables into FEs • FEs then switch memory banks and invalidate the route cache CS Dept., Purdue University

  28. Conclusions • Makes 2 important contributions • Emphasizes on examining every header: improves robustness and security • Shows it is feasible to build routers that can serve in emerging high-speed networks • In all, an excellent paper providing complete and intricate details about high speed router design CS Dept., Purdue University

More Related