Programmable Routers - PowerPoint PPT Presentation

Programmable routers
1 / 32

  • Uploaded on
  • Presentation posted in: General

Programmable Routers. Jae Woo Lee. Fundamental router design. Router. Routing protocols. RIB. Control plane. FIB. Packet forwarding. Forwarding plane (aka data plane). Software router. routed, OSPFd, GNU Zebra, Quagga, XORP. User-level daemons. OS kernel & Network devices.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Programmable Routers

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Programmable routers

Programmable Routers

Jae Woo Lee

Fundamental router design

Fundamental router design


Routing protocols


Control plane


Packet forwarding

Forwarding plane

(aka data plane)

Software router

Software router

routed, OSPFd,

GNU Zebra, Quagga,




OS kernel &

Network devices

Linux, BSD,



Extensible software control plane xorp

Extensible software control plane: XORP

  • Compete with Cisco & Juniper, and be extensible!

    • All standard protocols

    • Event-driven, not scanner-based

    • Multi-process architecture

    • Modern software engineering

  • Main contributions:

    • Staged design for BGP, RIB

    • Scriptable inter-process communication mechanism

    • Dynamically extensible CLI and management software

    • Extensible policy framework

Handley, M., Kohler, E., Ghosh, A., Hodson, O., and Radoslavov, P

Designing extensible IP router software, NSDI 2005

Conventional router implementation

Conventional router implementation

Slide borrowed from

Programmable routers



Slide borrowed from

Bgp staged architecture

BGP Staged Architecture

Slide borrowed from





  • Unmodified routes stored at ingress

    • Changes in downstream modules (filters, nexthop state, etc) handled by PeerIn pushing the routes again.




Filter Bank

Peer In


Slide borrowed from

Programmable routers


Routing Information Base


Slide borrowed from

Rib structure

RIB Structure

Routing protocols can register interest in tracking changes

to specific routes.

Slide borrowed from

Programmable routers


Interprocess communication


Slide borrowed from

Xrl xorp resource locator

transport: eg x-tcp, x-udp, kill, finder

module name: eg bgp, rip, ospf, fea

interface name: eg bgp, vif manager

typed parameters to method

method name: set_bgp_as, delete_route, etc

XRL: XORP Resource Locator

  • URL-like unified structure for inter-process communication:

  • Example:


  • Finder resolves to a concrete method instance, instantiates transport, and performs access control.


Slide borrowed from

Commercializing xorp vyatta

Commercializing XORP: Vyatta

  • Standard x86 hardware

  • Flexible deployment

    • Standard server hardware platforms

    • Blades

    • Virtualization

  • Open-source software

  • Why Vyatta is Better than Cisco,


  • Will an open source router replace your Cisco router?


Software forwarding plane os kernels

Software forwarding plane: OS kernels

Control plane

Interface between control and forwarding planes:

  • Linux (old)

    • /proc, sysctl, ioctl

  • Linux (new)

    • Netlink socket

  • BSD

    • Routing socket


routing daemons




routing socket

Linux kernel

Forwarding plane

  • J. Salim, H. Khosravi, A. Kleen, A. Kuznetsov, Linux Netlink as an IP Services Protocol, RFC 3549, July 2003

  • Bolla, R. and Bruschi, R., Linux Software Router: Data Plane Optimization and Performance Evaluation, Journal of Networks (JNW) 2, 3 (June 2007)

  • Qing Li, Kip Macy, Optimizing the BSD Routing System for Parallel Processing, PRESTO 2009

Modular software forwarding plane click modular router

Modular software forwarding plane:Click modular router

Control plane

  • Elements

    • Small building blocks, performing simple operations

    • Instances of C++ classes

  • Packets traverse a directed graph of elements




routing daemons

Linux kernel


Forwarding plane

  • Kohler, E., Morris, R., Chen, B., Jannotti, J., Kaashoek, M. F., The click modular router, ACM Trans. Comput. Syst. 18, 3 (Aug. 2000)

  • Andrea Bianco, Robert Birke, Davide Bolognesi, Jorge M. Finochietto, Giulio Galante, Marco Mellia, Click vs. Linux: Two Efficient Open-Source IP Network Stacks for Software Routers, HPSR 2005



PATS Research Group



Push and pull

Push and pull

Push connection

Source pushes packets downstream

Triggered by event, such as packet arrival

Denoted by filled square or triangle

  • Pull connection

    • Destination pulls packets from upstream

    • Packet transmission or scheduling

    • Denoted by empty square or triangle

  • Agnostic connection

    • Becomes push or pull depending on peer

    • Denoted by double outline

PATS Research Group



Push and pull violations

Push and pull violations

PATS Research Group



Implicit queue v explicit queue

Implicit queue v. explicit queue

  • Implicit queue

  • Used by STREAM, Scout, etc.

  • Hard to control

  • Explicit queue

  • Led to push and pull, Click’s main idea

  • Contributes to high performance

Ip router configuration

IP router configuration

PATS Research Group



Click performance circa 2000

Click performance, circa 2000

MLFFR with 64-byte packet:

333k, 284k, 84k for

Click, Linux w/ polling driver, Plain Linux

Improving software router performance exploiting parallelism

Improving software router performance:exploiting parallelism

  • Can you build a Tbps router out of PCs running Click?

    • Not quite, but you can get close

  • RouteBricks: high-end software router

    • Parallelism across servers and cores

    • High-end servers: NUMA, multi-queue NICs

    • RB4 prototype

      • 4 servers in full mesh acting as 4-port (10Gbps/port) router

      • 4  8.75 = 35Gbps

    • Linearly scalable by adding servers (in theory)

  • Dobrescu, M., Egi, N., Argyraki, K., Chun, B., Fall, K., Iannaccone, G., Knies, A., Manesh, M., and Ratnasamy, S. RouteBricks: exploiting parallelism to scale software routers, SOSP 2009

  • Bolla, R. and Bruschi, R., PC-based software routers: high performance and application service support, PRESTO 2008

Improving software router performance specialized hardware

Improving software router performance:specialized hardware


Network processor

  • Jad Naous, Glen Gibb, Sara Bolouki, Nick McKeown, NetFPGA: Reusable Router Architecture for Experimental Research, PRESTO 2008

  • Spalink, T., Karlin, S., Peterson, L., and Gottlieb, Y., Building a robust software-based router using network processors, SOSP 2001

  • J. Turner, P. Crowley, J. Dehart, A. Freestone, B. Heller, F. Kuhms, S. Kumar, J. Lockwood, J. Lu, M.Wilson, C. Wiseman, D. Zar, Supercharging PlanetLab – A High Performance, Multi-Application, Overlay Network Platform, SIGCOMM 2007

  • Tilman Wolf, Challenges and applications for network-processor-based programmable routers, IEEE Sarnoff Symposium, Princeton, NJ, Mar. 2006

Commercial hardware router juniper

Commercial hardware router:Juniper

Control plane

Routing Engine (RE)

  • RE

    • x86 PC running JUNOS

  • PFE

    • ASIC hardware and microcode

  • MS-PIC

    • MIPS64-based XLR network processor

    • Each runs separate JUNOS


    • FreeBSD-based OS for all Juniper routers

Switch Control Board (SCB)

Packet Forwarding

Engine (PFE)







Forwarding plane

Extending commercial router junos sdk

Extending commercial router:JUNOS SDK

  • RE SDK

    • Servers and management daemons running on RE

  • Services SDK

    • Data path apps running on MS-PIC

    • Packet processing with zero-copy API at line rate

    • 32 (virtual) CPUs

      • 8 cores  4 hardware threads

      • Data threads bound to dedicated CPUs to eliminate context switch

  • James Kelly, Wladimir Araujo, Kallol Banerjee, Rapid Service Creation using the JUNOS SDK, PRESTO 2009

Standardizing backplane ietf forces wg

Standardizing backplane:IETF ForCES WG


| | | | | | |

|OSPF |RIP |BGP |RSVP |LDP |. . . |

| | | | | | |


| ForCES Interface |


^ ^

ForCES | |data

control | |packets

messages| |(e.g., routing packets)

v v


| ForCES Interface |


| | | | | | |

|LPM Fwd|Meter |Shaper |NAT |Classi-|. . . |

| | | | |fier | |


| FE resources |


Examples of CE and FE functions.

  • Forwarding and Control Element Separation (ForCES)

  • Protocols for (multiple) control elements (CE) and forwarding elements (FE)

  • Separation can be switch fabric or LAN

  • Interoperability between router components

  • Would Cisco & Juniper care?

  • J. Salim, H. Khosravi, A. Kleen, A. Kuznetsov, Linux Netlink as an IP Services Protocol, RFC 3549, July 2003

  • H. Khosravi, Ed., T. Anderson, Ed., Requirements for Separation of IP Control and Forwarding, RFC 3654, November 2003

  • L. Yang, R. Dantu, T. Anderson, R. Gopal, Forwarding and Control Element Separation (ForCES) Framework, RFC 3746, April 2004

  • Ran Giladi, Niv Yemini, A programmable, generic forwarding element (GFE) approach for dynamic network functionality, PRESTO 2009

Control plane detached openflow

Control plane detached: OpenFlow



  • Physical separation of control and forwarding

  • Forwarding plane in L2

    • Flow table instead of FIB

    • More general than IP

  • Switch exposes flow table though simple OpenFlow protocol

    • Keep it simple

    • Vendor can keep platform closed

    • Use outboard device for packet processing




Flow table


Layer-2 Switch

Matches subsets of packet header fields





















  • McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., Shenker, S., and Turner, J., OpenFlow: enabling innovation in campus networks, SIGGCOMM Comput. Commun. Rev. 38, 2 (Mar. 2008)

Slicing network virtualization

Slicing network: virtualization

  • NIC virtualization

    • Solaris Crossbow

  • Router virtualization

    • Cisco & Juniper logical routers

    • Virtual Routers on the Move (VROOM)

Virtual router

Virtual router

Virtual router

  • Tripathi, S., Droux, N., Srinivasan, T., and Belgaied, K., Crossbow: from hardware virtualized NICs to virtualized networks, VISA 2009

  • Eric Keller, Evan Green, Virtualizing the Data Plane through Source Code Merging, PRESTO 2008

  • Yi Wang, Eric Keller, Brian Biskeborn, Jacobus van der Merwe, Jennifer Rexford, Virtual routers on the move: Live router migration as a network-management primitive, SIGCOMM 2008

Extreme programmability active networks

Extreme programmability:Active networks

Discrete approach:

code installed


  • Heated debate in the 90s

  • Far-reaching vision, still relevant today

Integrated approach:

packet carries code


  • Calvert, K., Reflections on network architecture: an active networking perspective, SIGCOMM Comput. Commun. Rev. 36, 2 (Apr. 2006)

  • David L. Tennenhouse, Jonathan M. Smith, W. David Sincoskie, David J. Wetherall, and Gary J. Minden, A Survey of Active Network Research, IEEE Communications Magazine, Vol. 35, No. 1, January 1997

  • David L. Tennenhouse, David J. Wetherall, Towards an active network architecture, SIGCOMM Comput. Commun. Rev. 26, 2 (Apr. 1996)

Hosting tomorrow s in network services netserv

Hosting tomorrow’s in-network services: NetServ

  • Reviving active network vision

    • Signaling-based code installation

    • Latest isolation and virtualization technology

    • Ubiquitous common API, from cable modem to Cisco router

  • Suman Srinivasan, Jae Woo Lee, Eric Liu, Mike Kester, Henning Schulzrinne, Volker Hilt, Srini Seetharaman, Ashiq Khan, NetServ: Dynamically Deploying In-network Services, ReArch 2009

Netserv prototype


Java OSGi on top of Click

Click: Modular router platform

OSGi: dynamic loading and unloading of modules


Bare Linux vs. Plain Click

Penalty for kernel-user transition

Plain Click vs. NetServ

Java overhead

2) is small compared to 1)

NetServ - prototype

Thank you

Thank you

  • Login