ACEs

1. INF5060:Multimedia data communication using network processors ACEs 24/9 - 2004

2. ACE • Active Computing Element • Defined by Intel’s SDK • Not part of hardware • Also sometimes called Action/Classification Engine • Idea behind ACEs • Element in a graph • Part of a classification engine

3. Classification

4. Ethernet Header IP Header TCP Header IP Header TCP Header Payload TCP Header Payload Payload Demultiplexing • Packet demultiplexing • Used with layered protocols • Packet proceeds through one layer at a time • On input, software in each layer choose module at next higher layer • On output, type field in each header specifies encapsulation Applications TCP UDP Payload IP MAC

5. Ethernet Header IP Header TCP Header Payload Classification • Packet classification • Alternative to demultiplexing • Designed for higher speed • Considers all layers at the same time • Linear in number of fields • Two possible implementation options • Software • Hardware Applications TCP UDP IP MAC … Payload

6. Classification • Idea • Classifiers perform a logical AND between fields • Hardware classifiers can perform AND operation in parallel • Reality • Need detailed distinctions involving many fields • Fields are not always at the same offset • Requirements for classification change over time

7. Classification • Need detailed distinctions involving many fields • Multiple categories • Web traffic • Telnet traffic • Mail traffic • ICMP traffic • other traffic

8. … .local etype immed[etype, ETH_IP] alu_shf[ --, etype, -, $$hdr3, >>16] br!=0[NotWeb#] .endlocal … br!=byte[$$hdr5, 0, IPT_TCP, NotWeb#] … .local wprt immed[wprt, TCP_WWW] alu[--, dport, -, wprt] br!=0[NotWeb#] .endlocal … .local base boff dpoff dport ld_field_w_clr[dpoff, 0001, $$hdr3, >>6] .local mask immed[mask, 0x3c] alu [ dpoff, dpoff, AND, mask ] .endlocal alu[dpoff, dpoff, +, 16] Buf_GetData[base, dl_buffer_handle] DL_GetBufferOffset[boff] alu[boff, boff, +, dpoff] alu_shf[boff, --, B, boff, >>3] sdram[read, $$hdr0, base, boff, 1], ctx_swap alu[ dpoff, dpoff, AND, 0x7 ] alu[ --, dpoff, -, 4] br>=0[SecondWord#] ld_field_w_clr[dport, 0011, $$hdr0, >>16] br[GotDstPort#] SecondWord#: ld_field_w_clr[dport, 0011, $$hdr1, >>16] GotDstPort#: Classification 0 8 16 24 32 40 48 56 64 dest src (0-1) src (2-4) type vers hlen service total length identification flgs frag. offsett ttl type header checksum src. address dest. address (0-1) dest. address (2-3) src. port dest. port seq. number (0-1) seq. number (2-3) acknowledgement vers reserv. code data window checksum urgent pointer possibleIP Options !! Logical AND …

9. Classification • Need detailed distinctions involving many fields • Multiple categories • Web traffic • Telnet traffic • Mail traffic • ICMP traffic • other traffic • Fields are not always at the same offset • Variable-sized headers • Fields not at fixed offsets • Easily handled with software • Finite cases can be specified in rules • Hybrid classification can be considered (hardware first, software next)

10. Hardware classifier Software classifier … Classification • Desirable • Fixed headers • Protocols without any optional fields • XTP • Options in dedicated locations • IPv6 – options follow header • Multiple classification stages

11. Classification • Static Classification • Flow specified in rule sets • Header fields and values know a priori • Dynamic classification • “Flows” created by observing the packet stream • Values taken from headers • Allows fine-grained flows • Requires state information

12. Version IHL DS Total length Identification D M Fragment offset Time to live Protocol Header checksum Source address Destination Address Padding Options (0 or more) Data 0 0 • Class selector codepoints • If of the form xxx000 • Differentiated Services Codepoint • xxxxx0 reserved for standardization • xxxx11 reserved for local use • xxxx01 open for local use, may be standardized later Example static classification • Allocate one flow per service type • DiffServ • One header field used to identify flow • IP ToS field / IP priority and DS fields DiffServ-enabled Router DiffServclassifiers Routing table

13. Dynamic classification • Allocate flow per TCP connection • Header fields used to identify flow • IP source address • IP destination address • TCP source port number • TCP destination port number • Applications • Clustered web servers • Load leveller dispatching TCP connection to several web servers • Firewalls • Allow incoming packets only when outgoing connection was made • More complicated firewall cases • FTP data connection • Voice over IP • RTP video streaming • …

14. Dynamic classification • Flow identification • By observing the packet stream • Packets belonging to the flow • FTP, SSH, … • Packets outside the flow • RSVP, IGMP, … • Connection-oriented network • Per-flow SVC can be created on demand • Flow ID equals connection ID • State • Connection • Allocated resources • Connectionless network • Flow ID used internally • Each flow ID mapped to (next hop, interface) • State • Next hop, interface mapping

15. Dynamic classification • Dynamic classification • Usually performed in software • State kept in memory • State information created/updated at wire speed

16. ACEs

17. ACE Features • Active Computing Element • Defined by Intel’s SDK • Not part of hardware • Also sometimes called Action/Classification Engine • One ACE: one step in dynamic classification • ACEs in Intel’s SDK • Fundamental software building block • Runs on StrongARM, microengine, or host • Handles control plane and fast or slow data path processing • Coordinates and synchronizes with other ACEs • ACEs form a graph • Used to construct packet processing systems • Can have multiple inputs or outputs • Can serve as part of a pipeline

18. ACE Terminology • Microengine • One of the specialized processors of the IXP card • Microblock • Code that runs • in one thread • on one microengine • MicroACE • Core component runs on StrongARM • Microblock component runs on microengines • Source microblock • Handles ingress from I/O device • Sink microblock • Handles egress to I/O device • Transform microblock • Intermediate position in a pipeline

19. Four conceptual parts of an ACE • Initialization • Classification • Actions associated with each classification • Message and event management

20. Initialization Classification Actions associated with each classification Message and event management source/init.c … ix_error ix_init(int argc, char **argv, ix_ace ** ap) { struct wwbump *wwb; ix_error e; (void)argc; *ap = 0; wwb = malloc(sizeof(struct wwbump)); if ( wwb == NULL ) … wwb->ue = atoi(argv[2]); strcpy(wwb->bname, "WWBUMP"); wwb->name[sizeof(wwb->name) - 1] = '\0'; strncpy(wwb->name, argv[1], sizeof(wwb->name) - 1); e = ix_ace_init(&wwb->ace, wwb->name); if (e) … e = RmInit(); if (e) … e = RmRegister(&wwb->tag, wwb->bname,&wwb->ace, exception, wwb, wwb->ue); if (e) … e = cc_init(wwb); if ( e ) … *ap = &wwb->ace; return 0; } Four conceptual parts of an ACE

21. Initialization Classification Actions associated with each classification Message and event management ucbuild/WWBump.uc … .local etype immed[etype, ETH_IP] alu_shf[ --, etype, -, $$hdr3, >>16] br!=0[NotWeb#] .endlocal … br!=byte[$$hdr5, 0, IPT_TCP, NotWeb#] … .local wprt immed[wprt, TCP_WWW] alu[--, dport, -, wprt] br!=0[NotWeb#] .endlocal … Four conceptual parts of an ACE

22. Initialization Classification Actions associated with each classification Message and event management source/action.c … ix_error exception(void *ctx, ix_ring r, ix_buffer b) { struct wwbump *wwb = (struct wwbump *) ctx; ix_error e; unsigned char c; (void) r; e = RmGetExceptionCode(wwb->tag, &c); if ( e ) … Webcnt++; e = RmSendPacket(wwb->tag, b); if ( e ) … return 0; } Four conceptual parts of an ACE

23. Initialization Classification Actions associated with each classification Message and event management source/wwbcc.c ix_error getcnt(ix_base_t* bp, long* rv) { (void)bp; *rv = Webcnt; return 0; } Four conceptual parts of an ACE

24. ACE has set of outputs Not in wwbump – only default Each output given target name Outputs bound dynamically at run time Unbound target corresponds to packet discard ixsys.config-wwbump … microace ifaceInput ./ingressAce none 0 1 microace ifaceOutput ./egressAce none 1 2 microace wwbump ./wwbump none 0 0 … bind static ifaceInput/default wwbump bind static wwbump/default ifaceOutput … Output targets and late binding

25. ACE has set of outputs Create other targets in source/init.c Each output given target name Outputs bound dynamically at run time Unbound target corresponds to packet discard SDK Reference ix_error ix_target_init( ix_target* targp, ix_ace* acep, const char* name); Targp – where to store the new target handle Acep – ACE to which the new target belongs Name – symbolic name of the new target Output targets and late binding

26. Conceptual ACE interconnection • Ingress ACE acts as source • Process ACE acts as transform • Egress ACE acts as sink • Connections created at run time ixsys.config-wwbump … bind static ifaceInput/default wwbump bind static wwbump/default ifaceOutput …

27. Single ACE with two components StrongARM (core component) Microengines (microblock component) Communication possible between components arm-be/wwbump SlowIngressWWBump.uof ucbuild/WWBump.uc … immed[dl_next_block, IX_EXCEPTION] br[Finish#] … ucbuild/WWB_dl.uc … alu[ --, dl_next_block, -, IX_EXCEPTION] br=0[Send_SA#] … Send_SA#: DL_SASink[ ] .continue … source/action.c … ix_error exception(void *ctx, ix_ring r, ix_buffer b) Components of MicroACE

28. Communication: Exceptions • Packets passed from microblock to core component • Mechanism • Microcode set dl_next_block to IX_EXCEPTION • Dispatch loop forwards packet to core • ACE tag used to identify corresponding component • Exception handler is involed in core component

29. Consume the packet and free the buffer Modify the packet before sending it on Send the packet back to the microblock for further processing Forward the packet to another ACE on the StrongARM SDK Reference ix_error ix_buffer_del(ix_buffer* buf) source/action.c ix_error exception(void *ctx, ix_ring r, ix_buffer b) { … e = RmSendPacket( wwb->tag, b); … } SDK Reference ix_error ix_target_take(ix_target* targp, ix_buffer* buf); targp – bound target to pass to buf – buffer to pass to target Actions applied to exceptions

30. Division of ACE into components • Microblocks form fast path

31. Illustration of microblock groups • Set of one or more microblocks • Treated as single unit • Loaded onto one microengine for execution • Can be replicated on microengines for higher speed • Entire microblock group assigned to same microengine

32. Illustration of microblock groups • Set of one or more microblocks • Treated as single unit • Loaded onto one microengine for execution • Can be replicated on microengines for higher speed ucbuild/WWB_dl.uc … DL_Init[] EthernetIngress_Init[] WWBumpInit[] .while(1) Top_Of_Loop#: DL_SASource[ ] alu[--, dl_buffer_handle, -, IX_BUFFER_NULL] br=0[Test_Ingress#], guess_branch br[Send_MB#] Test_Ingress#: EthernetIngress[ ] alu[--, dl_buffer_handle, -, IX_BUFFER_NULL] br=0[Top_Of_Loop#] … Dispatch loop WWBump microblock SlowIngress microblock

33. Microblock structure • Asynchronous model • Neither core ACE nor MicroACE may block • Programmer creates • Initialization macro • Dispatch loop

34. Event loop • Central to asynchronous programming model • Uses polling • Repeatedly checks for presense of event(s) and calls appropriate handler • Can be hidden from programmer • In ACE model • Explicit • Programmer can modify/extend • Note • Event loops are also used when programming Berkeley sockets with select()

35. Event loop processing • If event loop stops • All processing stops • The arrival of a new event will not trigger invocation of an event handler • Conclusion: the event loop must go on

36. Crosscall mechanism • Used between • Core component of one ACE and another • ACE core component and non-ACE application • Not intended for packet transfer • Operates like Remote Procedure Call • Mechanism know as crosscall

37. Both caller and callee programmed to use crosscall Not dynamic Not at run-time Declaration given in Interface Definition Language (IDL) IDL compiler Read specification Generates stubs that handle calling details include/idl/wwbump.idl interface wwbump { twoway long getcnt(); }; include/wwbump_cb_c.h include/wwbump_cc_c.h include/wwbump_import.h include/wwbump_sk_c.h include/wwbump_stub_c.h Crosscall implementation

38. Crosscall implementation • Three types of crosscalls • Deferred: caller does not block; return notification asynchronously • Oneway: caller does not block; no value returned • Twoway: caller blocks; callee returns a value • Twoway call corresponds to traditional RPC • Type of call determined at compile time • Note • ACEs are not allowed to block • Therefore an ACE can not call a twoway crosscall (only answer one)

39. Packet queues • Packet queues • Generated by the SDK when ACEs communication asynchronously • Buffer packets • Placed between ACE components • Permit asynchronous operation

40. Illustration of packet queues

41. Packet queues • Operation • ix_target refers to the ring used to transfer packets from the current ACE to another ACE • The target may be bound or not • If bound, buffers to the target (using ix_target_take) are placed in the ring • If unbound, buffers to the target are discarded • ix_res_bind allocates a new ring for a target • ix_res_unbind disassociates current packet processing queue from the target • all previously taken packets will still processed by the target ACE

42. Summary • Intel SDK uses ACE programming model • ACE • Basic unit of computation • Can include code for StrongARM (core) and microengines (microblock) • Packet queues used to pass packets between ACEs • Crosscall mechanism used for nonpacket communication