An NP-Based Router for the Open Network Lab

1. An NP-Based Router for the Open Network Lab Jon Turnerwith Patrick Crowley, John Dehart, Brandon Heller, Fred Kuhns, Jing Lu, Mike Wilson, Charlie Wiseman, Dave Zar

2. Objectives for ONL Router • Reproduce approximately same functionality as current hardware router • routes, filters (including sampling filters), stats, plugins • Extensions • multicast, explicit-congestion marking • Use each NPU as separate 5 port router • each responsible for half the external ports • xScale on each NPU implements CP functions • access to control variables, memory-resident statistics • updating of routes, filters • interaction with plugins through shared memory • simple message buffer interface for request/response

3. ONL NP Router xScale add configurable per port delay (up to 150 ms total delay) xScale add largeSRAM ring TCAM SRAM HdrFmt (1 ME) Rx (2 ME) Parse, Lookup, Copy (3 MEs) Mux (1 ME) QueueManager (1 ME) Tx (2 ME) largeSRAM ring Stats (1 ME) • Each output has common set of QiDs • Multicast copies use same QiD for all outputs • QiD ignored for plugin copies Plugin Plugin Plugin Plugin Plugin SRAM xScale largeSRAM ring

4. Unicast, ARP and Multicast • Each port has Ethernet header with fixed source MAC address – several cases for destination MAC address • Case 1 – unicast packet with destination on attached subnet • requires ARP to map dAdr to MAC address • ARP cache holds mappings – issue ARP request on cache miss • Case 2 – other unicast packets • lookup must provide next-hop IP address • then use ARP to obtain MAC address, as in case 1 • Case 3 – Multicast packet • lookup specifies copy-vector and QiD • destination MAC address formed from IP multicast address • Could avoid ARP in some cases • e.g. point-to-point link • but little advantage, since ARP mechanism required anyway

5. Proposed Approach • Lookup does separate route lookup and filter lookup • at most one match for route, up to two for filter (primary, aux) • combine route lookup with ARP cache lookup • xScale adds routes for multi-access subnets, based on ARP • Route lookup • for unicast, stored keys are (rcv port)+(dAdr prefix) • lookup key is (rcv port)+(dAdr) • result includes Port/Plugin, QiD, next-hop IP or MAC address, valid next-hop bit • for multicast, stored keys are (rcv port)+(dAdr)+(sAdr prefix) • lookup key is (rcv port)+(dAdr)+(sAdr) • result includes 10 bit copy vector, QiD • Filter lookup • stored key is IP 5-tuple + TCP flags – arbitrary bit masks allowed • lookup key is IP 5-tuple + flags if applicable • result includes Port/Plugin or copy vector, QiD, next-hop IP or MAC address, valid next-hop bit, primary-aux bit, priority • Destination MAC address passed through QM

6. Lookup Processing • On receiving unicast packet, do route & filter lookups • if dAdr on local subnet (determined by port’s IP address/mask) • if MAC address returned by route (or higher priority primary filter) is valid, queue the packet and continue • else, pass packet to xScale, marking it as no-MAC • leave it to xScale to generate ARP request, handle reply, insert route and re-inject packet into data path • else • if lookup returns valid MAC, queue packet and continue • else, send to xScale with no-MAC bit set • On receiving multicast packet, do route & filter lookups • take higher priority result from route lookup or primary filter • format MAC multicast address • copy to queues specified by copy vector • if matching auxiliary filter, filter supplies MAC address