ESP: A Language for Programmable Devices

1. ESP: A Language for Programmable Devices Sanjeev Kumar Princeton University Advisor : Kai Li

2. D D CPU R R CPU Mem Mem D D Programmable Devices Main CPU Main Memory Bus Network Card Disk Network Move functionality from main CPUs to devices ESP: A Language for Programmable Devices

3. Programming Devices • Challenging • Concurrent, low-overhead, and reliable • Event-driven state machines (vs. Threads) • 1 stack is shared by all state-machines • Fast context switches • Only need to save and restore the Program Counter • Low overhead: memory & CPU • Less functionality provided ESP: A Language for Programmable Devices

4. WaitReq WaitDMA WaitNet event DmaFree execute event NetFree execute Programming Devices in C • State machines communicating using events • A state machine eventUserReqexecuteprocessUser() event KernelReq execute processKernel() ESP: A Language for Programmable Devices

5. Programming Devices in C Cont’d • Programming is hard • Sequential code split into handlers • Difficult for C compiler to optimize • Explicitly save values in global variables • Hand optimizations make it worse • Fast paths (violate modularity) • Embed ad-hoc scheduling decisions int i = 5; block(); j = i; ESP: A Language for Programmable Devices

6. High-performance communication Bypass OS for data transfers Used Myrinet network cards Gigabit network 33 MHz CPU, 1 MB memory VMMC firmware Event-driven state machines in C Case Study: VMMC Application Data OS Network Card Network ESP: A Language for Programmable Devices

7. Our Experience with Programming VMMC Firmware • Event-driven state machines in C • Hard to program • VMMC: 15,600 lines of C code • Add new functionality • Hard to debug • Complicated race conditions (still encounter bugs) • Trusted by the OS • Achieves good performance • Requires hand optimization ESP: A Language for Programmable Devices

8. Outline • Motivation • ESP overview • ESP language • Verifying ESP programs • Performance of ESP programs • Conclusions ESP: A Language for Programmable Devices

9. ESP: Event-driven State-machines Programming • Domain-specific language • Goals • Easy to program • Concise, modular programs • Allow extensive testing • Use existing model-checking verifiers like Spin • Performance • Low overhead • Case study: VMMC firmware ESP: A Language for Programmable Devices

10. Develop and Test using Verifier pgm.C help.C pgm.ESP pgm1.spin test1.spin ESP Compiler Generate Firmware pgmN.spin testN.spin ESP Approach Spin Models Detailed Model Memory Safety Model Abstract Models ESP: A Language for Programmable Devices

11. Related Work • Verification + Code generation • Esterel: Control of reactive systems • Teapot: Coherence protocols for shared memory • Promela++: Layered network protocols • @-format: C extension for state machines • Concurrent languages • CSP, Squeak, OCCAM, Java, CML, Devil • Debugging Tools • Meta-level compilation, SLAM, Verisoft ESP: A Language for Programmable Devices

12. Outline • Motivation • ESP overview • ESP language • Verifying ESP programs • Performance of ESP programs • Conclusions ESP: A Language for Programmable Devices

13. The ESP Language • Concurrent language • Processes & Channels • in : Receive a message on a channel • out : Send a message on a channel • alt : Wait on multiple in/out operations • Channels are synchronous or unbuffered • Processes and channels are static ESP: A Language for Programmable Devices

14. channelchan1: int channelchan2: int process add5 { $v: int = 0; while( true) { in( chan1, v); out( chan2, v+5); } } States are implicit Events are messages on channels Easy to read & optimize Control flow is obvious Initial waitChan1 waitChan2 A Process Encodes a State Machine ESP: A Language for Programmable Devices

15. Data Types and Control Constructs • Data types • Simple types: int, bool • Complex data types: records, unions, arrays • No recursive data types • Control constructs • if-then-else, while, etc. • No functions; Use processes instead ESP: A Language for Programmable Devices

16. Channels for Communication • Processes communicate only using channels • Pure message passing communications • To enforce this • No shared mutable data structure • No global variables • Only immutable objects on channels • Immutable mutable objects ESP: A Language for Programmable Devices

17. Explicit malloc/free Fast Unsafe Automatic Garbage collection More CPU & memory Safe Concurrent program Safe Limited CPU & memory Fast Memory Management ESP: A Language for Programmable Devices

18. Memory Management Cont’d • ESP supports explicit management • Explicit management is difficult • Global property of program • ESP makes this a local property • Objects are not shared by processes • Objects sent over channels are “copied” • Use verifiers to check correctness • Check each process separately • ESP supports safety through verification ESP: A Language for Programmable Devices

19. channelpktc: union of { ping: int, status: bool } process A { ...; out( pktc, v); ...} processB { ...; in( pktc, { ping |> $node}); ...} processC { ...; in( pktc, { status |> $flag}); ...} Support for Dispatch Pattern-matching supports dispatch ESP: A Language for Programmable Devices

20. External Interface • C interface: volatile memory, device registers • Spin interface: specify properties to be verified • Traditional approach: function interface • ESP uses channels • Some channels have external reader or writer • Unified mechanism (C and Spin) • Use in/out to block on external events ESP: A Language for Programmable Devices

21. Case Study: VMMC Firmware • Implemented VMMC using ESP • 8 processes, 19 channels • 500 lines ESP + 3000 lines C code • Modular programs that are easy to maintain • Order of magnitude less code ESP: A Language for Programmable Devices

22. Outline • Motivation • ESP overview • ESP language • Verifying ESP programs • Performance of ESP programs • Conclusions ESP: A Language for Programmable Devices

23. Using Model-Checking Verifiers • State-space exploration • Try all possible scheduling options • Advantages • Automatic • Produces counter example • Disadvantages • Computationally expensive (exponential) • ESP currently uses Spin model checker ESP: A Language for Programmable Devices

24. Spin Model-Checking Verifier • Designed for software systems • Supports processes and synchronous channels • Specify properties to be verified • Assertions, deadlocks, Linear Temporal Logic • 3 levels of checking with varying coverage • Exhaustive • Partial • Simulation ESP: A Language for Programmable Devices

25. Develop and Test using Verifier pgm.C help.C pgm.ESP pgm1.spin test1.spin ESP Compiler Generate Firmware pgmN.spin testN.spin ESP Approach • Models extracted automatically • Reduces programmer effort • Avoids mismatch • Debugged using verifier • Test files can be reused ESP: A Language for Programmable Devices

26. Extracting Spin Models • Detailed models • Memory-safety models • Detailed model + additional checks • Abstract models • Necessary for larger subsystems • Drop unnecessary details • Depending on the property being verified • Abstraction specified by the programmer • Compiler makes safe conservative approximation ESP: A Language for Programmable Devices

27. X if :: b1 = true :: b1 = false fi X type recT = #record of { int count; } X $r1: recT = {0}; if (b) { r2 = r1; } ... r1.count = 5; X if :: r2.count = 5 :: skip fi X Conservative Approximationsin Abstract Models $b2: boolean = true; ... $b1:boolean= b2; ESP: A Language for Programmable Devices

28. Case Study: VMMC Firmware • Develop and debug retransmission protocol • Easier to debug than on the network card • Took 2 days (10 days in earlier implementation) • Check for memory safety and leaks • Found deliberately introduced bugs, an earlier bug • Check for deadlocks • Hard-to-find bugs • Found 7 bugs using abstract models ESP: A Language for Programmable Devices

29. Case Study: VMMC Firmware Cont’d * Using partial mode ESP: A Language for Programmable Devices

30. Outline • Motivation • ESP overview • ESP language • Verifying ESP programs • Performance of ESP programs • Conclusions ESP: A Language for Programmable Devices

31. ESP Code Generation • Compiling for sequential execution • Combine them [Berry et al., Proebsting et al.] • No context switching overhead • Worst-case exponential increase in executable size • Low-overhead process management • Small context switching overhead • Only the program counter needs to be saved • Small executable ESP: A Language for Programmable Devices

32. ESP Code Generation Cont’d • C as back-end language • Generates one large C function • Links with other functions provided by the programmer • Compiler optimizations • Whole program analysis • Avoid indirect jumps, unnecessary allocation • Per process • Constant folding, copy propagation ESP: A Language for Programmable Devices

33. Case Study: VMMC Firmware • Measure overhead of using ESP • Microbenchmarks • Applications • Compare performance • Earlier implementation (vmmcOrig) • Earlier implementation without fast paths (vmmcOrigNoFastPaths) • New implementation using ESP (vmmcESP) ESP: A Language for Programmable Devices

34. Latency s Message size (in Bytes) Microbenchmarks • Impact of fast paths • 4 Bytes: 100% • 4 Kbyte: 38% • Impact of using ESP • 64 Bytes: 35% • 4 Kbytes: 0% ESP: A Language for Programmable Devices

35. Applications Speedup SPLASH2 applications on a 16 (4 X 4) node cluster Fast paths: < 1% on average ESP: 3.5 % on average ESP: A Language for Programmable Devices

36. Performance Summary • Significant difference in microbenchmarks • Most due to the brittle fast paths • Required the programmer to manually optimize • Small impact on applications • Applications are less sensitive • Microbenchmarks represent worst case • New functionality can help a lot more • 40% for SVM applications [ Bilas et al. ] ESP: A Language for Programmable Devices

37. Outline • Motivation • ESP overview • ESP language • Verifying ESP programs • Performance of ESP programs • Conclusions ESP: A Language for Programmable Devices

38. Conclusions • ESP: Domain-specific language • Supports concise, modular programs • Order of magnitude less code • Extensive testing with Spin • Develop code • Check for memory safety & absence of deadlocks • Effective but has limitations • Low performance overhead • 3.5% for applications • Could use further optimizations ESP: A Language for Programmable Devices

39. Future Directions • More experience with ESP • Other devices e.g. Intel IXP • Other protocols e.g. TCP/IP • Compiler optimizations • Selective process inlining • Support for fast paths • Using model checking more effectively • Partial searches ESP: A Language for Programmable Devices

40. Acknowledgements Kai Li Andrew Appel Edward Felten Randy Wang Larry Peterson Doug Clark Jaswinder Pal Singh Steve Dirk Rudro Patrick Yuanyuan Tammo George Angelos Stef Liviu Others Melissa and the rest of administrative staff Technical staff ESP: A Language for Programmable Devices

41. - The End - ESP: A Language for Programmable Devices

42. Esterel • Synchronous programming language • Encode control of reactive systems • Generate software or hardware • Deterministic reaction • But • Only encodes control flow • Constraints on the language • Harder to compile efficiently ESP: A Language for Programmable Devices

43. Teapot • Domain-specific language • Implement coherence protocols • Specify a state machine • Use continuations to simplify programming • But • Only encodes control flow • Uses handlers: Code fragmentation • Similar approach would be less modular ESP: A Language for Programmable Devices

44. Promela++ • Language for Layered Network Protocol • Non-strict extension of promela • Asynchronous communication • But • Layered structure • Asynchronous: overhead, ordering • No support for memory management ESP: A Language for Programmable Devices

45. Bandwidth Microbenchmarks One-way Bandwidth Bidirectional Bandwidth M B / s M B / s Message Size (in Bytes) Message Size (in Bytes) ESP: A Language for Programmable Devices

46. Fast Paths • Challenges • Identify fast paths in a concurrent program • Fast path extraction • Robust fast paths ESP: A Language for Programmable Devices