Issues and Trends in Router Design

1. Issues and Trends in Router Design Presented by: Sudarshan Krishnaprasad August 28th, 2003

2. Contents • Introduction • Components of a router • Anatomy of a router • Network processor • Functionalities of IP routers • Route processing • Packet forwarding • Route look up • Special functions of a router • Router Architecture • IP Packet Processing in a Distributed Router Architecture • Switching Fabric • Conclusion • References

3. Introduction • What are routers? • Determines the route your packet takes in travel from the source to destination and also routes them in that direction. • When does routing take place? • Sending to an IP address • IP packet comes to the router • What does the router do? • The router determines where the destination is and how it should be reached by consulting its routing tables • Why is it difficult? • Heterogeneous link technologies • Scheduling support for differential service. • Participate in complex distributed algorithms to generate routing tables • Insatiable need for bandwidth

4. Introduction • The three different routing classes are: • Access routers: These allow homes and small businesses to connect to the internet service provider • Enterprise routers: They link tens and thousands of computers within a campus or an enterprise • Backbone routers: They link together ISP’s and enterprise networks with a long distance trunks.

5. Components of a Router • Input Port: • It is the point of attachment of physical link and is the point of entry for incoming packets • Switching fabric: • This interconnects the input ports and the output ports. • Input queued router and output queued router. • Output port: • Stores the packets and schedules them for service on an output link • Routing Processor: • Participates in routing protocols and creates a forwarding table that is used for packet forwarding.

6. Components of a Router • Line interfaces: physically attach multiple transmission systems to the node and provide framing functionality • Network processors: provide the intelligence and processing power to analyze packet headers, look up routing tables, classify packets based on their destination and source addresses and other control information and (often complex) rules, and provide queuing and policing of packets • Switch Fabric: which provides high-speed (ideally non-blocking) interconnection of the node’s packet processing units • System Processor: which performs control point functions such as route computation and box and network management.

7. Components of a Router • Network Processors • Fast, economical, but rarely flexible enough for rapid adaptation to protocol or standard changes. • Will implement performance critical packet forwarding functions in software • The bit stream processors receive the serial stream of packet data and extract the information needed to process the packet, (IP source/destination address, type of service (TOS) bits, or TCP source/destination port numbers) • The packet is then written into the packet buffer memory • The extracted control information is fed to the processor complex, which constitutes the programmable unit of the NP.

8. if needed, extracts additional information from the packet and submits the relevant part to the search engine, looks up the medium access control (MAC) or IP address, classifies the packet, or does a virtual circuit/path identifier (VCI/VPI) lookup if the packet is recognized as an asynchronous transfer mode (ATM) cell using the routing and bridging tables and appropriately designed hardware assists. Based on the results returned, the processor instructs the scheduler to determine the appropriate departure time of the packet. Upon packet transmission through the bit stream processor, the necessary modifications to the packet header are performed. Components of a Router

9. Functionalities of a Router • Generally consists of several network interfaces to the attached networks, processing modules, buffering modules, internal interconnection networks • Generic IP router architecture as shown: • Interface cards perform the inbound and outbound packet forwarding • CPU in router performs functions like path computations, routing table maintenance, and reach ability propagation. • Basically, • Route processing • Packet forwarding • Router special services

10. Functionalities of a Router

11. Route Processing • Routing protocols are the means by which routers gain information about the network. Routing protocols map network topology and store their view of that topology in the routing table. E.g.: Routing Information Protocol and OSPF • Routing table consists of entries that specify the destination and next hop router through which the datagram should be sent. • Construction and maintenance using routing protocols • Route calculation consists of determining the route: network, subnet, network prefix or host. • Static routing • Routing table entries are created by default (bootstrap file, ICMP) • Tables cannot be configured once created • Cannot reroute across a disabled link and alarms • Network manager maintains the routing table • Basically for small networks, there is a single connection point to other networks, no redundant routes.

12. Functionalities of a Router • Dynamic routing • Used for internetworking across wide area networks • Automatically reconfigures the routing table and recalculates the least expensive path • Routers broadcast packets to signify their presence and communicate, cost of connections and load levels • Reconfiguration must be quick without incorrect information • The routing table is updated dynamically • If multiple routes are found, the protocol chooses the best route and bypass the problems of broken links. • Two principal routing protocol types • Interior gateway protocols (IGP’s) within an autonomous system eg: RIP, OSPF and IS-IS • Exterior gateway protocols (EGP’s) between autonomous systems Eg: Border Gateway Protocol (BGP)

13. Packet Forwarding • Forwarding Process • Packet forwarded from I/p to O/p based on contents of IP • Packet’s networks layer remains unchanged, data-link layers and physical transmission schemes change radically • Checks for errors and sees If entire packet is received, if malformed entire packet is discarded • Verifies the TTL and makes sure the packet does not circulate forever, If TTL reaches 0, ICMP message sent to host, router changes only TTL and checksum • Unicast, multicast and broadcast packets • Broadcast storm for unicast packets • Destination IP used to lookup routing tables, indicating the interface to forward out of and IP of the next router. Link address is determined and prepended in the IP. • Large Packet is divided into fragments • ARP (Address resolution protocol) converts the IP of the next hop to a data-link, and similar process is repeated if necessary

14. Route Lookup • Looking up a route in routing table (finding the longest prefix match) is a performance bottleneck • The address classes A, B, and C (allowing sites to have 24, 16, and 8 bits respectively for addressing) proved too inflexible and wasteful of the resources, and Class C addresses were given out which increased the tables massively. • Classless Inter-Domain routing was introduced which aggregated the addresses where advertised as a whole instead of individual addresses. • Routers have a default route (0.0.0.0) usually overridden, eg: if one router connects to the internet then all routers use this default route to connect to the internet. • The longest prefix match used radix trees or modified particia trees • These lookup alogirithms have complexity based on the number of bits, 32 for IPv4 Performance depends on data, might have to backtrack • Some routers use route cache to cache the recent addresses to avoid the routing table (not suitable for backbone routers)

15. Route Lookup • Typically 2 types of packets arrive at the router, • packets to the router or • packets forwarded to the network. • Check the packets explicitly if it belongs to the router, saves time by not going to the massive routing table. • The recent algorithms for routing table lookups use small and compressed data structure using a small amount of memory. Hence result in faster lookups

16. Special Functions of Routers • Authentication and access services (Packet filtering for security). • Routers often used between company networks and the internet to prevent unauthorized access • Foils most attacks and makes it easy to back track (ISP). • Also used in congestion control. (Discadring packets algorithm) • Used to define traffic statistics (ISP to implement usage based charging schemes)

17. Router Architecture • Processing power, memory bandwidth and internal bus bandwidth are main bottle necks. • Routers are mainly dedicated systems not running any specific application tasks • Every network interface is equipped with proper processing power and buffer space to solve the processing power problem • internal bus bandwidth can be solved with an effective switch, allowing simultaneous transfers between network interfaces. • To cope with high aggregate packet rates, we need efficient processing and memory components, switch based router architecture is used in most commercial routers for this reason.

18. Router Architecture • Each network interface provides the processing power and the buffer space needed for packet processing tasks related to all the packets flowing through it. • They perform specific tasks and offer QoS, depending on the protocol the port specifies the packets to predefined classes • The exact features of the processing components depend on the functional partitioning and implementation details. • Interfaces interconnected via a high performance switch and a CPU is used to perform centralized tasks • The MSI performs physical layer functions and SFI is responsible for preparing the protocol on its next hop • Based on the kind of functionality desired by the protocol there are two paths involved.

19. Router Architecture • Critical data path processing (fast path) • The processing tasks directly related to packets being forwarded through the router can be referred as the time critical processing tasks. • These processing tasks comprise all protocols involved in the critical path (e.g., Logical Link Control, (LLC) Sub network Access Protocol ( SNAP) and IP) as well as ARP which can be processed in the network interface because it needs direct access to the network, even though it is not time critical. • The time critical tasks mainly consist of header checking, and forwarding (and may include segmentation) functions. • Generally, the fast path of IP routing requires the following functions: • IP packet validation, destination address parsing and table lookup, packet lifetime control (TTL update), and checksum calculation. • While they are not trivial, it is possible to implement them in hardware, thereby providing performance suitable for high-speed routing.

20. Router Architecture • Non critical data path processing (slow path) • Packets destined to a router, such as maintenance, management or error protocol data are usually not time critical. • Typical examples of these non-time critical processing tasks are error protocols (e.g., ICMP), routing protocols (e.g.,RIP, OSPF, BGP), and network management protocols (e.g., SNMP). • These processing tasks need to be centralized in a router node and typically reside above the network or transport protocols. • Network interface implements all IP packet forwarding protocols and all others are by the CPU.

21. Router Architecture

22. IP packet process • IP header validation: forwarding logic verifies all layer 3 information (header length, packet length, protocol version, checksum etc) • Route lookup and header processing: Looks up the IP destination to lookup output port and performs forwarding operations (TTL decrement, checksum increment etc) • Packet classification: forwarding engine examines layer 4 and higher layer packet attributes relative to QoS and access control • Performs the following functions in parallel: • Associates the packet to the right output port • Redirects the packet to a different destination • Drops the packet in case of congestion, or security policy and • Performs the appropriate accounting functions • The forwarding engine notifies the system controller that packet has arrived. • The system controller reserves a memory location or the arriving packet

23. IP Packet Process 7. Once packet is in shared memory, system controller assigns the appropriate o/p port. 8. O/P port extracts the packet from shared memory 9. Once packet is received in O/P port memory location is made available for new traffic.

24. Switching Fabric • Responsible for transferring packets from the input modules to the appropriate output modules • Different popular approaches: • Shared memory, output queued. • Shared medium, output queued. • Distributed output buffered, output queued (commercially used) • Space division switch fabric: the crossbar switch (commercially used)

25. Switching Fabric • Shared medium switch fabric: • Packets routed by means of a shared medium e.g.: bus, ring or dual bus, bus is the simplest • Used TDM to share the medium • Incoming packets are sequentially broadcasted on the bus, filters at the O/P determine if the packet is destined to that O/P • If packets are more than the port speeds, Input queuing is used. • Normally used for broadcast approach.

26. Switching Fabric • Shared memory switch fabric • Packets are converted from serial to parallel form and written sequentially into a dual port RAM. • Outgoing packets are de-multiplexed and then converted from parallel to serial again. • Output buffer approach with a centralized buffer for all ports. Main advantage of statistical sharing • Must write into memory at least once and memory access time imposes a limit on packet transfer. • Multicasting not a natural approach. Packet must be in memory until read by all multicast ports.

27. Switching Fabric • Distributed output buffered switch fabric • Independent paths exist between all N2 possible pairs of inputs and outputs. In this design, arriving packets are broadcast on separate buses to all outputs. • Address filters at each output determine if the packets are destined for that output. • Appropriate packets are passed through the address filters to the output queues • No conflict among the I/p and o/p ports, output queuing approach, multicast is natural • All hardware operates at the same speed.

28. Switching Fabric • Used in most designs as no speed up factor is present. • Unfortunately, the quadratic N2 growth of buffers means that the size N must be limited for practical reasons • The port speed S can be increased to the physical limits of the address filters and output buffers. • Hence, this approach might realize a high total throughput NS packets per second.

29. Switching Fabric • Space division switch fabric: the crossbar switch • This architecture appears to be especially convenient for providing QoS guarantees, • serious limitations are : • the switch fabric must operate at a rate at least equal to the aggregate of all the input links connected to the switch. • One of the most popular interconnection networks used for building input buffered switches is the crossbar because of its (i) low cost, (ii) good scalability and (iii) non-blocking properties. • In this architecture buffering occurs at the inputs, and the speed of the memory does not need to exceed the speed of a single port. • Given the current state of technology, this architecture is widely considered to be substantially more scalable than output buffered or shared memory switches, but presents a lot of technical challenges, eg HOL • One way of reducing the effect of HOL blocking is to increase the speed of the input/output channel

30. Switching Fabric • Another way is VOQ, Instead of maintaining a single FIFO at the input, a separate queue per each output can be maintained at each input. • Thus a combination of input buffered and output buffered switch is required, i.e., Combined Input and Output Buffered (CIOB). • Here, the shared buffer provides a repository for the heads of all input queues and hence serves as a contention resolver.

31. Switching Fabric

32. Conclusion IP provides a high degree of flexibility in building large and arbitrary complex networks. Internetworking routers capable of forwarding aggregate data rates at terabits per second will be the need for performance network environment. We mainly discussed the architecture and design overview of the components that have highest effect on performance. Significant advances have been made in router design to address the most demanding needs and promise to scale to the necessary data rates and system sizes. A system and software structure that optimally distributes both packet by packet and control processing tasks among the processors will be crucial for a network router.

33. References • Issues and Trends in Router Design: • S. Keshav and R. Sharma Cornell University • On the Design of IP routers: Router Architectures • James Aweya Nortel Networks • Technologies and Building Blocks for Fast Packet Forwarding • Werner Bux et. Al IBM