Optimizing Compilers CISC 673 Spring 2011 Dynamic Compilation

1. Optimizing CompilersCISC 673Spring 2011Dynamic Compilation John Cavazos University of Delaware

2. High Level View of JVM

3. JVM Interpreter • Reads a bytecode from a method • “Interprets” the bytecode • Decodes opcode and operands • Based on opcodes jumps to some C code • Passes operands • Continues reading bytecodes from method until: • Call • Return • Exception

4. Interpretation • Popular approach for high-level languages • Ex, Python, APL, SNOBOL, BCPL, Perl, MATLAB • Useful for memory-challenged environments • Low startup time & space overhead, but much slower than native code execution • MMI (Mixed Mode Interpreter) [Suganauma’01] • Fast interpreter implemented in assembler

5. Dynamic Compilation Techniques • Baseline compiler • Translates bytecodes one by one to machine code • Quick compilation • Reduced set of optimizations for fast compilation

6. Dynamic Compilation Techniques • Full compilation • Full optimizations only for selected hot methods • Classic just-in-time compilation • Compile methods to native code on first invocation • Ex, ParcPlace Smalltalk-80, Self-91 • Initial high (time & space) overhead for each compilation • Precludes use of sophisticated optimizations (eg. SSA) • Responsible for many of today’s myths

7. Interpretation vs JIT Execution: 20 time units Execution: 2000 time units

8. Selective Optimization Hypothesis: most execution is spent in a small percentage of methods (90/10 rule) Idea: use two execution strategies 1. Interpreter or non-optimizing compiler 2. Full-fledged optimizing compiler Strategy: • Use option 1 for initial execution of all methods • Profile to find “hot” subset of methods • Use option 2 on this subset

9. Selective Optimization Selective opt: compiles 10%-20% of methods, representing 90-99% of execution time Execution: 20 time units Execution: 2000 time units

10. Designing a Selective Optimizer • AKA: Adaptive Optimization System • What is the system architecture? • What are the profiling mechanisms and policies for driving recompilation? • How effective are these systems?

11. Basic Structure of a Dynamic Compiler Still needs good core compiler - but more Machine code Program Structural inlining unrolling loop perm Scalar cse constants expressions Memory scalar repl ptrs Reg. Alloc Scheduling peephole

12. Executing Program Program Basic Structure of a Dynamic Compiler Instrumented code Raw Profile Data History prior decisions compile time Optimizations Profile Processor Interpreter or Simple Translation Processed Profile Compiler subsystem Compilation decisions Controller

13. Method Profiling • Counters • Call Stack Sampling • Combinations

14. Method Profiling: Counters • Insert method-specific counter on method entry and loop back edges • Counts how often a method is called and approximates how much time is spent in a method • Very popular approach: Self, HotSpot • Issues: overhead for incrementing counter can be significant • Not present in optimized code

15. Method Profiling: Counters foo ( … ) { fooCounter++; if (fooCounter > Threshold) { recompile( … ); } . . . }

16. Method Profiling: Call Stack Sampling • Periodically record which method(s) on call stack • Approximates amount of time spent in each method • Can be compiled into the code • Jikes RVM, JRocket • or use hardware sampling • Issues: timer-based sampling is not deterministic

17. A B C Method Profiling: Call Stack Sampling A A A A A B B B B ... ... C C Sample

18. Method Profiling Mixed • Combinations • Use counters initially and sampling later on • IBM DK for Java foo ( … ) { fooCounter++; if (fooCounter > Threshold) { recompile( … ); } . . . } A B C

19. Recompilation Policies Problem: given optimization candidates, which should be optimized? • Counters: Optimize method that surpass threshold • Simple, but hard to tune, doesn’t consider context • Sampling: Optimize method on call stack top • Addresses context issue

20. Recompilation Policies Problem: given optimization candidates, which should be optimized? • Call Stack Sampling: • Optimize all methods that are sampled • Simple to implement • Use cost/benefit model • Seemingly complicated, but easy to engineer • Maintenance free • Naturally supports multiple optimization levels

21. Jikes RVM: Recompilation Policy – Cost/Benefit Model • Define • cur, current opt level for method m • Exe(j), expected future execution time at level j • Comp(j), compilation cost at opt level j • Choose j > cur that minimizes Exe(j) + Comp(j) • If Exe(j) + Comp(j) < Exe(cur) recompile at level j

22. Jikes RVM: Recompilation Policy – Cost/Benefit Model • Assumptions • Sample data determines how long a method has executed • Method will execute as much in the future as it has in the past • Compilation cost and speedup are offline averages

23. Optimization Levels Optimization Level Optimizations Controlled Branch Opts Low Constant Prop / Local CSE Reorder Code Opt Level O0 Copy Prop / Tail Recursion Static Splitting / Branch Opt Med Simple Opts Low Opt Level O1 While into Untils / Loop Unroll Branch Opt High / Redundant BR Simple Opts Med / Load Elim Expression Fold / Coalesce Global Copy Prop / Global CSE SSA Opt Level O2

24. Short Running Programs No FDO, Mar’04, AIX/PPC

25. Short Running Programs No FDO, Mar’04, AIX/PPC

26. Steady State No FDO, Mar’04, AIX/PPC

27. Steady State

28. Profiling for What to Do • Myth: Sophisticated profiling is too expensive to perform online • Reality: Well-known technology can collect sophisticated profiles with sampling and minimal overhead

29. Suggested ReadingDynamic Compilation • Adaptive optimization in the Jalapeno JVM, M. Arnold, S. Fink, D. Grove, M. Hind, and P. Sweeney, Proceedings of the 2000 ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages & Applications (OOPSLA '00), pages 47--65, Oct. 2000.

30. Spare Slides

31. Method Profiling Timer Based if (flag) handler(); • Useful for more than profiling • Jikes RVM • Schedule garbage collection • Thread scheduling policies, etc. if (flag) handler(); class Thread scheduler (...) { ... flag = 1; } void handler(...) { // sample stack, perform GC, swap threads, etc. .... flag = 0; } foo ( … ) { // on method entry, exit, & all loop backedges if (flag) { handler( … ); } . . . } A if (flag) handler(); B C

32. Arnold-Ryder [PLDI 01]: Full Duplication Profiling • Generate two copies of a method • Execute “fast path” most of the time • Execute “slow path” with detailed profiling occassionally • Adapted by J9 due to proven accuracy and low overhead