Runtime Feedback in a Meta-Tracing JIT for Efficient Dynamic Languages

1. Runtime Feedback in a Meta-TracingJIT for Efficient Dynamic Languages Writer: Carl Friedrich Bolz Introduced by Ryotaro IKEDA at 2011/09/06

2. Overview This paper describes about… How to make it more efficient to apply JIT compiler with PyPy PyPy : Well-known as fast Python implementation. However, in actual, it is one of framework to implement interpreter with JIT and GC! ( Python implementation is just a demo! ) What is PyPy? Framework which enables to write interpreter implementation with Restricted Python The project mainly intends to give environments to implement dynamic interpreter much efficient

3. PyPy’s JIT Automatic Implementation Architecture It is implemented by PyPy user! Target code that is written in any language Any interpreter that is written in RPython Give some “hints” to enable to run JIT compiler efficiently Run PyPy’sRPython interpreter The most bottom one performs JIT compilation and optimization to the middle one In result, JIT compiler that is suitable for any language is automatically implemented

4. How to treat non-language-specific JIT compilation Typical JIT Compiler Uses language-specific feature because each JIT compiler is dedicated to compile only one language PyPy’s JIT Compiler Though it is for RPython, PyPy can’t use any language-specific featurewhich PyPy user want to implement. It is what we called “ Meta-Tracing “ How can we make it much faster with applying efficient method for Meta-Tracing?? = Objective

5. What Merit Using PyPyRather than JIT of other implementation Widen compilation / optimization area It is too challenging for JIT compiler to target data structure operation Typical JIT Implementation It traces ,and only looks to whole RPython code, so it can target data structure operation which written in RPython by developer. PyPy’s JIT Implementation

6. Hinting Mechanism Main concept Code Hint Giving hints to enable JIT compiler to compile efficiently is the most important RPython PyPy MAIN HINTS A hint to turn arbitrary variables into constants in the trace by feeding back runtime information into compilation A way to annotate operations which the constant folding opti- mization then recognizes and exploits. General techniques for refactoring code to expose constant folding opportunities of likely runtime constants. ☆ ☆ ☆

7. PyPy’s Meta-Tracing JIT Compilers Tracing To check and determine which control path to compile x = 100 y = 200 Cycle: Trace Cond Optimizations are also performed during this trace form Op Op x = x + y Also constant-folded Trace (cycles, to be compiled): Cond -> x = x + 200 -> Cond ….

8. PyPy’s Tracer Trace Area PyPy (can / by default) traces only “hot” paths. -> Trace will be invoked frequently executed path Counter It indicates how many times the loop is executed When it crosses threshold, it is regarded as “hot” 1000 for x in sequence : t = x + …. … … ☆ As mentioned before, PyPy’s tracer doesn’t trace user program directly, but interpreter implementation written in RPython instead.

9. Optimization Passes • Remove/simplify operations in the trace • Constant folding • Common subexpression elimination • Allocation removal • Store/load propagation • Loop invariant code motion These can be applied because traces are absolutely linear form Operate duringRPython form

10. Running Example Arrangements of shown examples Simple and bare-bones object model. • Just supports classes and instances • No inheritance • Class contains method and variable • Instance have a class, if no requested method / variable found in the instance, it searches among the class.

11. Example Implementation Use dictionary to manage class method Use dictionary to manage instance attributes(variables/methods) To search requested method To register given method Dictionary’s “get” method costs too much. To solve the problem, it is required to make it target to JIT compilation ( The way to do this is described later discussion )

12. Hints for Controlling Optimization Applied only to interpreter written in RPython, not user program. • Two hints that enables to increase the optimization opportunities for constant folding Enable propagation to find “Constant- foldable” variables via trace guard Promotion Trace-Elidable Annotation to notify which variables are assumed as constant variable ☆ Though each of them never break code’s behavior, Using them incorrectly will definitely deteriorate its speed.

13. What “Guard” is Dynamic Language That both x and y are number, or string is OK test = x + y; Static Language That both x and y are either number or string, types cannot be canged test = x + y; “Guard” It is necessary to assure each variables’ type are same to compile Dynamic Language to Static Language Native code is one of static language, it’s needed

14. How “Guard” works Guard assures that interpreter is running compiled trace in same condition as when it is compiled at first time. Source code Trace result Now it Becomes hot! y = 10 z = 100 for x in sequence: x = y + z y += 1 …. = func(x) guard(x == int() ) guard(y = int()) guard(z = 100) x = y + z y += 1 …. = func(x) Assure conditions to compile them to machine code 100 If conditions described in guard is true, it continues to run. During execution of compiled machine code… If conditions described in guard is false, it stops to run and switch to interpreter exec.

15. Promotion • Technique to operate constant-fold using guard Result after Promotion Source Code Trace Result P R O M O T E x = somefunc() y = func(x) x = somefunc() guard( x == 200) y = func(200) x = somefunc() guard( x == 200) y = func(x) Trace tree x = somefunc() TRUE guard(x == 200) FALSE y = func(200) y = func(x) Normal execution root (interpreter) Traced root (will be / already compiled)

16. “Promote” how to Use “promote()” embedded method which is given by PyPyRPython interpreter to give it a “hint” that indicates “promote” can be applied during this scope. Assume the trace here usually be with a condition that self and val are expected to not so frequently varied. Guard-fall is expected not so occurs frequently Later discussion! (Soon!) It may not consume overhead so much and can be expected that constant-folding will bring great improvement.

17. “Trace-Elidable” helps to apply “Promote” • To tell the truth, promote cannot be invoked without @elidable annotation in the example. Trace-Elidable: Assure specific method never change any variables. Though tracer want to “promote” method “f” , tracer doesn’t know whether self.c() returns always same value or not… Tracer considers not to use value-specific guard but type-specific guard… never “promoted”. @elidable annotations shows that given method is immutable This “hint” enablestracer to promote f()!

18. Result trace after these 2 hints applied COMMON Before After Constant-folding is applied via @elidable and promote. This trace is created without any hints given.

19. Technique to increate “trace-elidable” Putting It All Together Increasing the amount of Trace-elidable method increases chance to apply constant-folding and to help Promote. for index map (described in next slide) Prepare original “Map” class to manage Instance’s attributes instead of using dictionary To append @elidable annotations!

20. Index map • Efficient / Suitable data structure for PyPy Map: To manage data location (index) “string” : 1 “v1” : 0 “x” : 3 List: Stores actual data Prepare getindex with this impelemntation, though it is immutable, trace-elidable can be used!

21. How does Instance use the “Map”? This class which is used for manage instances no longer uses dictionary! No longer use dictionary Whole methods belong to “map” are “trace-elidable”. So the promote will work correctly!

22. Versioning of Classes Using only trace-elidabledon’t satisfy requirements In Python, though @elidable annotation is given, the method may yield not same value because any attributes can be changed. How do you feel if “inst.x = -1” is executed? class A: def __init__(self): x = 100 @elidable def X(self): return x inst = A() It is necessary to handle this possible changes They propose “Versioning”

23. Use Guard Feature to Versioning Dummy class to use guard feature This promote helps to create value-specified guard with current “version”. So, it is still trace-elidable but can handle methods changing. When some of methods is changed, Yield new VersionTag and save it to self.version

24. Evaluations Environment: Intel Core2 Duo P8400 processor with 2.26 GHz and 3072 KB of cache on a machine with 3GB RAM running Linux 2.6.35 It uses many OOP’s features No hints given Algorithm for board game BZ2 decoder OS Kernel Simulation Decimal floating Point calculations

25. Conclusions • Two hints that can be used in the source code of an interpreter written with PyPy. • They give control over runtime feedback and optimization to the language implementor. • They are expressive enough for building well-known virtual machine optimization techniques, such as maps and inlining.

26. Effects to my Study • Use PyPy as infrastructure • It can emit C source code from RPython implementation • Applying P.T seems easy • Parallelized Template for Rpython • This paper performs optimizations in RPython form.How do you think that I consider to implementtemplate code in RPython?