The Performance of µ-Kernel-Based Systems

1. The Performance of µ-Kernel-Based Systems H. Härtig, M. Hohmuth, J. Liedtke, S. Schönberg and J. Wolter Dresden U. and IBM TJ Watson Appears in SOSP 1997 Presented by: Fabián E. Bustamante

2. Customizability Research in OS • Goal • Provide flexible mechanisms and policies to clients • Different context – different goals for customization (perf., power consumption, conflicting client needs, …) • Design issues • Performance – while sometimes introduced to improve perf., customizability mechanisms have a cost • Spread – throughout the OS or just part of it (such as the scheduling policy) • Granularity – all aspects of a policy or in some restricted manner (such as a menu of options scheduling policies) • Integrity – clearly, level and complexity of what needs to be enforced is related to both granularity and spread • Purpose – special- or general-purpose OS? • Paradigm – a traditional, posix-like OS or something new (such as Scout data-paths)

3. Taxonomy • A high-level taxonomy* – criteria • Initiator of adaptation • Human admin or OS designer • Application • OS itself • Time of adaptation • Design, build or installation (static) • Boot or run-time (dynamic) *Denys, Piessens and Matthijs, “A Survey of Customizability in Operating Systems Research”, ACM Computing Surveys, 34(4):450-468, Dec. 2002.

4. Historical Context • Kernel • The mandatory part of an OS • Can use all features of a processor • Most early OS were monolithic • Complete OS was packed into a single kernel (scheduling, file system, memory management,…) • Microkernel approach • Minimize kernel, implementing servers on top • Ideally, only ASs, IPC & basic scheduling is left inside

5. Software Technology Advantages • Different APIs, FS, even different OS strategies can coexists • Flexibility and extensibility • Server malfunction is isolated, even device drivers • Clean µk i/f enforces a more modular system structure • Smaller kernel is easier to maintain and less prone to error • Smaller Trusted Computing Base (hw, µk, disk drive and maybe the FS)

6. First Generation & Evaluation Criteria • Conceptual breakthrough to µks • Mach’s external pager – k manages physical and virtual memory; forwards page faults to user-level task • Handling hardware interrupts as IPCs & include I/O ports into AS – k captures the interrupt but doesn’t handle it (interrupt handling and device I/O done outside k) • Appealing yes, but is it useful? Are the concept flexible & costs low enough? • Critical mechanism for efficiency - IPC • Stabilized in Mach 3 – a Mach RPC 10x Conventional Unix • Conclusion – the layer of abstraction provided by µks is either • too low - look at extensible-kernel ideas, figure out how to protect kernels from misbehaved extensions (SPIN, Vino, etc) or • too high – exterminate all abstractions; OS’ two jobs – protection (safely multiplex physical resources) and provide an abstract machine - forget the latter (exokernel)

7. Observation • Problem is measurements • The system you measure – legacy problem with 1st generation µks • How you measure it and what conclusions you draw - is it the approach, the concepts implemented by a particular µk or the implementation of the µk? • Evaluation criteria then • Conservative criterion – applications must not be degraded by µk - compatibility performance • Progressive criterion - µk must efficiently support new types of apps w/ good performance - extensibility performance • Approach • A monolithic Unix kernel, Linux, was adapted to run on top of L4 (as a user-level single server) • Performance was compared to MkLinux, a third-party port of Linux to a Mach 3.0 derived µk (first generation) • Different application, generic and specific were implemented and evaluated • Performance portability checked w/ an Alpha-port of L4

8. L4 Essentials • Based on 2 basic concepts: • Threads – An activity executing inside an AS • AS – a mapping which associates each virtual page to a physical page frame • Recursive construction of ASs • By magic there’s a one AS σ0 • µk provides 3 operations • Grant – owner of an AS can grant any page to another AS • Map – owner can map any page into another AS, now both can access it • Flush/demap/… - onwer can flush any of its pages w/o warning • All AS are constructed and maintained by user-level servers – pagers • Threads can dynamically associate individual pagers with themselves • I/O ports are treated as parts of ASs • Hardware interrupts are handled as IPC • Pentium-specific – small-AS (<512MB) can be physically shared & protected by Pentium’s segment mechanism

9. User process User process User process User process Linux Server Initial space (physical memory) Linux on Top of L4 • No-tuned port of Linux 2.0 • Goal – full binary compatibility • Double page tables (inside the kernel for security, and in the Linux server for portability)

10. Source-code Lines

11. Compatibility Performance • What’s the penalty of using L4Linux instead of native Linux • Run benchmarks on both in the same hardware • Does the performance of the underlying µk matter? • Compare L4Linux to MkLinux, both in user-level and as an extension Microbenchmark – getpid system call costs

12. More Microbenchmark Results

13. Macrobenchmarks • Compilation time for the Linux server

14. More Macrobenchmarks Uses the shared libc.so so it avoids overhead of trampoline

15. More Macrobenchmarks For native Linux – maximum load of 130 jobs per minute

16. Extensibility Performance • E.g. Application-level Pipes – shows that the pipe performance can be increased drastically when using a user-level pipe implementing usingµk facilities • Linux – standard pipe provided by Linux, running native • L4Linux – same on L4Linux w/ shared library • L4 … app-level – runs on bare L4 • Restricted – synchronous L4 RPC (blocking IPC w/o buffering)

17. Extensibility Performance • Cache partitioning • Run a 64x64-matrix multiplication • Interrupted by a synthetic load that maximizes cache conflicts • Uninterrupted – 10.9ms • Interrupted every 100 microsec, worst-case 96.1 ms • Using cache-partitioning, enabled by user-level pagers, allocating 3 secondary-cache pages (of 64) – worst-case 24.9ms

18. Conclusion • L4Linux ~ native Linux – 5-10% penalty • Collocation on its own is not sufficient to fix performance problems • What about L4 w/ collocation? • Pipes and some VM operations used as examples of improve Unix-compatible functionality • Real-time memory management example shows the advantage to coexisting systems based on other paradigms

19. General Conclusions • Applying the performance criterion to such complex systems is complex – naïve, uninterpreted measurements are sometimes misleading • Early µk measurement suggested reducing the IPC cost, but the real problem was the structure and implementation of kernels • Although steady evolution is a powerful methodology, sometimes a radical approach is needed • Most problems of 1st generation µk were caused by their step-by-step development