Running A First Example: The Web Bumper

1. INF5060:Multimedia data communication using network processors Running A First Example:The Web Bumper 3/9 - 2004

2. Overview • Packet header and encapsulation review • How to use the card • programming model • booting • starting and stopping • Lab setup • Web bumper

3. Packet Headers and Encapsulation

4. Ethernet 48 bit address configured to an interface on the NIC on the receiver 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | Source address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Frame type | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 48 bit address configured to an interface on the NIC on the sender describes content of ethernet frame, e.g., 0x0800 indicates an IP datagram

5. Internet Protocol version 4 (IPv4) indication of the abstract parameters of thequality of service desired – somehow treat high precedence traffic as more important – tradeoff between low-delay, high-reliability, and high-throughput – NOT used, bits now reused indicates the format of the internet header, i.e., version 4 length of the internet header in 32 bit words, and thus points to the beginning of the data (minimum value of 5) datagram length (octets) includingheader and data - allows the lengthof 65,535 octets first zero, fragments allowed and last fragment 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Version| IHL |Type of Service| Total Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Identification |Flags| Fragment Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Time to Live | Protocol | Header Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Destination Address | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ identifying value to aid assembly of fragments indicate where this fragment belongs in datagram disable a packet to circulate forever,decrease value by at least 1 in each node – discarded if 0 checksum on the header only – TCP, UDP over payload. Since some header fields change(TTL), this is recomputed and verified at each point indicates used transport layer protocol 32-bit address fields. May be configured differently from small to large networks options may extend the header – indicated by IHL. If the options do not end on a 32-bit boundary, the remaining fields are padded in the padding field (0’s)

6. UDP port to identify the sending application port to identify receiving application 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Length | Checksum | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ specifies the total length of the UDP datagram in octets contains a 1’s complement checksum over UDP packet and an IP pseudo header with source and destination address

7. TCP code bits: urgent, ack, push, reset, syn, fin sequence number for data in payload acknowledgementfor data received port to identify the sending application port to identify receiving application 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Source Port | Destination Port | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Sequence Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Acknowledgment Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Header| |U|A|P|R|S|F| | | length| Reserved |R|C|S|S|Y|I| Window | | | |G|K|H|T|N|N| | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Checksum | Urgent Pointer | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Options | Padding | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ receiver’s buffer size foradditional data header length in 32 bit units pointer to urgent data in segment contains a 1’s complement checksum over UDP packet and an IP pseudo header with source and destination address options may extend the header. If the options do not end on a 32-bit boundary, the remaining fields are padded in the padding field (0’s)

8. Encapsulation

9. Using IXP1200

10. ingress ACE process ACE egress ACE output ports input ports Programming Model • Active computing elements (ACEs): • basic programming abstraction defined by Intel’s software developer kit (SDK), not part of hardware • asynchronous unit of execution • a typical system contains at least three in a pipeline: • unidirectional queues between ACEs allow processors to run independently • late bindings are used to bind ACE output at run-time(unbounded targets can be used to drop packets)

11. Programming Model • Packet flow illustration for IP forwarding: Stack ACE ingress ACE (core) IP ACE (core) egress ACE (core) StrongARM erroneous packet, not IP, … packet for this host microengine ingress ACE (microblock) IP ACE (microblock) egress ACE (microblock) output ports input ports packet is not for me, forward packet is an error free IP datagram

12. Programming Model • Often many possible communication paths, but only one used per packet • Division of work between ACEs running both on microengines and StrongARM crucial for high performance: • microengines comprise fast data path – the common case • StrongARM handles all exceptions

13. Programming Model • Challenge: achieve optimal parallelism • efficient resource utilization  each part of the pipeline should spend the same amount of time to process a packet, reduce idle time • which microengine runs which ACE, many possible configurations, e.g., • exactly one microblock ACE per engine • a pipeline of microblocks for ACEs on one or more engines • microblock groups: • set of microblocks that run on a single engine • specified in a configuration file

14. ingress ACE process ACE egress ACE output ports input ports Programming Model • Microblock group: • Replicated microblock groups microengine 1 microengine 2 ingress ACE process ACE input ports egress ACE microengine 1 output ports ingress ACE process ACE input ports microengine 3 microengine 2

15. Stack ACE ingress ACE (core) IP ACE (core) egress ACE (core) ingress ACE (microblock) IP ACE (microblock) egress ACE (microblock) Programming Model • Microblock dispatch loop • control packet flow among ACEs in a microblock group • runs in an infinite loop, e.g., IP forwarder:initialize microblocks while (1) { get next packet from input port invoke ingress microblock if not IP packet send to ingress core else { invoke IP microblock if packet not is for this host send to egress microblock else send to IP core }}

16. Programming Model • Exceptions (interrupts raised by microengines) • refer to packets passed from microblock components (microengine) to a core component (StrongARM) • the symbolic constant IX_EXCEPTION can be used • components are identified using a tag • an exception code informs the core component of the type • core component must have an exception handler determining how to process the packet

17. Programming Model • Crosscalls • enable an ACE to invoke a function in another ACE • similar to remote procedure calls (RPC) • must be programmed into both caller and callee • early binding, programmer determines type (not run-time) • specified by declaring a function • three types • deferred – caller do not block ; asynchronous return notification • oneway – caller do not block ; no return value • twoway – caller blocks until callee returns value

18. File Access, Communication and Booting • PCI Ethernet emulation • physically, the card plugs into the host’s PCI bus • host and board communicates sending Ethernet frames over the PCI • The disk is mounted using NFS over PCI, e.g.,/opt/ixasdk is available at both host and card • The card itself has no persistent memory • a RAM disk for the root file system is • when booting the card, an image is loaded from the host • bootixp performs these instructions and boots the card • allow some time after booting • Rebooting the card from the board (reboot) only clears/reinitialize the card  must run bootixp from host afterwards • Running bootixpon a running card will crash the card • boot (reinitialize) card through minicom, followed by a bootixp, or • host reboot

19. Environment Requirements • The Makefiles assume that the following environment variables are set: • IXROOT = /opt/ixasdk • CONFIG = arm_be • … this is already set for the root user on the lab machines

20. Starting and Stopping • Telnet to the card: telnet ixp • To start the example: ixstart <config_file> • our example config files use relative paths • cd $IXROOT/bin/arm_be ; ./ixstart ixsys.config • To stop the example: ixstop • stops whatever ACE that is running • cd $IXROOT/bin/arm_be ; ./ixstop

21. Lab Setup

22. Internet Lab Setup IXP lab switch switch ssh connection … Reboot Error Student lab switch power switch “reboot x”

23. Lab Setup – Power Switch IXP lab switch switch … power switch

24. IXP lab switch switch … power switch Lab Setup - Addresses

25. IXP lab switch switch … power switch Lab Setup - Addresses IXP lab 192.168.67.111 129.240.67.111 switch switch 192.168.67.112 129.240.67.112 192.168.67.5 192.168.67.113 129.240.67.113 … … … 192.168.67.118 129.240.67.118 power switch 129.240.67.110

26. ssh connection (129.240.67.112) IXP1200 Lab Setup – Data Path IXP lab PCI IO hub switch switch hub interface memory hub local network (192.168.67.112) … To 192.168.67.5 system bus CPU RAM interface To 192.168.67.5 memory power switch web bumper (counting web packets and forwarding all packets from one interface to another)

27. Web Bumper

28. ssh connection (129.240.67.112) IXP1200 Lab Setup – Data Path IXP lab IO hub switch switch memory hub local network (192.168.67.112) … CPU To 192.168.67.5 memory power switch web bumper (counting web packets and forwarding all packets from one interface to another)

29. IXP1200 The Web Bumper • the wwbump coreACE checks a packet forwarded by the microACE: • count web packet - add 1 to counter • send back to wwbump microACE • provides access to counter via a crosscall server • the wwbump microACE checks all packets ifit is a web packet: • if not, forward packet to egress microACE (and sent to the other port) • if web packet: • send to wwbump coreACE (StrongARM) • forward packets from core to egress microACE (and sent to the other port) • the wwbump microACE checks all packets ifit is a web packet: • if not, forward packet to egress microACE (and sent to the other port) • if web packet: • send to wwbump coreACE (StrongARM) • forward packets from core to egress microACE (and sent to the other port) web bumper wwbump (core) StrongARM microengines input port output port ingress ACE(microblock) wwbump(microblock) egress ACE(microblock) web bumper (counting web packets and forwarding all packets from one interface to another) ingress processing encompasses alloperations performed as packets arrive egress processing encompasses all operations applied as packets depart

30. IXP1200 The Web Bumper the crosscall client reads the web packet counter and prints the current count web bumper crosscallclient wwbump (core) StrongARM microengines input port output port ingress ACE(microblock) wwbump(microblock) egress ACE(microblock)

31. Configuration File • It is run at boot time and specifies • ACEs to be loaded • which side to run an ACE (ingress or egress) • number and speed of ports • see ixsys.config • The interfaces that are to be started at system boot time: interface 0 10.1.0.1 10.1.0.255 255.255.255.0 00:01:02:03:04:05 1interface 1 10.2.0.1 10.2.0.255 255.255.255.0 00:01:02:03:04:06 1… • Using workbench or not (we do not) mode 0

32. Configuration File • The microcode and microaces that are to be started at system boot timefile 0 ./SlowIngressWWBump.uof • ingress microcode (type 0) • associated slow ports (10/100 Mbps) microace ifaceInput ./ingressAce none 0 1 • ifaceInput is a microace • executable • no config file • running on ingress side • type ingress microace wwbump ./wwbump none 0 0 • runs on ingress side (first zero) • has unknown type (second zero) • …

33. Configuration File • Bind configurationbind static ifaceInput/default wwbump • binding for ingress (ifaceInput) is wwbump bind static wwbump/default ifaceOutput • binding for wwbump output is ifaceOutput • Shell commands to be run, e.g., add ARP entries in the cache

34. Identifying Web Packets • These are the header • fields you need for the web bumper: • Ethernet type 0x800 • IP type 6 • TCP port 80

35. WWBump Dispatch Loop initialize dispatch loop macrosdo forever { if packet has arrived from StrongARM send to egress if packet has arrived from Ethernet port { invoke WWBump macro to process packet if it is a web packet (exception specified) send to StrongARM else send to egress } }

36. WWBump Macro allocate 6 SDRAM registers to hold header data get IXP input port  set IXP output port to the otherread first 24 bytes into the 6 SDRAM registers (etype, IP hlen, IP type)if not frame type is IP (0x800), forward packetif not IP type is TCP (6), forward packetcalculate port number address using IP header lengthread TCP destination port numberif not TCP destination port is 80, forward packetset exception codeset wwbump tagset IX_EXCEPTION return

37. Testing • Log in on the assigned IXP machine (129.240.67.xxx) using ssh as root (twice, the example is easier using two different windows) • Download code and place at a convenient place under $IXROOT/src, e.g.: • cd $IXROOT/src/microaces/aces/ • scp –r user@login.ifi.uio.no:/hom/paalh/INF5060code/wwbump bumptest • Compile the code and make sure the targets are moved to $IXROOT/bin/arm_be • microcode for the microengines: cd$IXROOT/src/microaces/aces/bumptest/ucbuild; make ; cp *.uof $IXROOT/bin/arm_be/. • c-code for the StrongARM: cd .. ; make (wwbump and getcount are automatically put right directory) • Copy configuration file: cp ixsys.config$IXROOT/bin/arm_be/ixsys.config-bumptest • Telnet to the IXP card as root: telnet ixp • Enter bin directory: cd $IXROOT/bin/arm_be/ (you may already be there!!) • Stop any running aces: ./ixstop • Start the bumper: ./ixstart ixsys.config-bumptest

38. Testing The bumper should now be running: • In IXP window, check web packet count (should be 0): ./getcount • In HOST window • ping web server machine: ping 192.168.67.5(packets should be forwarded through the card, but not counted, i.e.: ./getcount 0) • telnet to web server machine using port 80: telnet 192.168.67.5 80(packets should be forwarded through the card, and counted, i.e.: ./getcount 4 (!!??)) Experiment with the card: • … start and stop the aces • … reboot the card • … remotely restart the machine using the power switch

39. Summary • We’ll use remote access to the IXP lab machines • Bumper shows the basic concepts • Intel SDK and programming model • microengines and StrongARM • much code even for a small, trivial task • Next: short demo (here) and testing (in the lab) • Next week: extending the bumper example