Investigating Adaptive Compilation using the MIPSpro Compiler

1. Investigating Adaptive Compilation using the MIPSpro Compiler Keith D. CooperTodd Waterman Department of Computer Science Rice University Houston, TX USA

2. Motivation • Despite astonishing increases in processor performance certain applications still require a heroic compiler effort • Scientific applications: weather, earthquake, and nuclear physics simulations • High quality compilation is difficult • The solutions to many problems are NP-complete • Many decisions that impact performance must be made • The correct choice can depend on the target machine, source program, and input data • Exhaustively determining the correct choices is impractical • Typical compilers use a single preset sequence of decisions • How do we determine the correct sequence for each context?

3. Adaptive Compilation • An adaptive compiler experimentally explores the decision space • Uses a process of feedback-driven iterative refinement • Program is compiled repeatedly with a different sequence of optimization decisions • Performance is evaluated using either execution or estimation • Performance results are used to determine future sequences • Sequence of compiler decisions is customized to always provide a high level of performance • Compiler easily accounts for different input programs, target machines and input data • Can current compilers be used for adaptive compilation?

4. Experimental Setup • Searched for certain properties in a compiler • Produces high quality executables • Performs high-level optimizations • Command-line flags that control optimization • Selected the MIPSpro compiler • Initial experiments showed that changing blocking sizes could improve running times • Loop Blocking • A memory hierarchy transformation that reorders array accesses to improve spatial and temporal locality • Major impact on array based codes • Includes DGEMM -- a general matrix multiply routine • Allows comparison with ATLAS

5. ATLAS • Automatically tuned linear algebra software • Goal is to achieve hand-coded performance for linear algebra kernels without a programmer modifying the code for each processor • Kernel is modified and parameterized once by a programmer • When ATLAS is installed on a machine experiments are run to determine the proper parameters for the kernel • Saves human time at the expense of additional machine time • Adaptive compilation aims to take this tradeoff one step further

6. Adjusting Blocking Size • Compare three versions of DGEMM • Compiled with MIPSpro and varying specified block sizes • Built by ATLAS • Compiled with MIPSpro using built-in blocking heuristic • Test machine: SGI MIPS R10000 • 195 MHz processor • 256 MB memory • 32 KB L1 data cache • 1 MB unified L2 cache

7. DGEMM running time for 500 x 500 arrays

8. DGEMM running time for 1000 x 1000 arrays

9. DGEMM running time for 1500 x 1500 arrays

10. DGEMM running times for square matrices

11. Relative DGEMM running times

12. L1 Cache Misses for DGEMM

13. L2 Cache Misses for DGEMM

14. Adjusting Blocking Size • The performance of MIPSpro using the built-in blocking heuristic drops off substantially when the array size reaches 900 x 900 • Far more L1 cache misses • Fewer L2 cache misses • Heuristic uses a rectangular blocking size that increases as the total array size increases • MIPSpro with adaptively chosen blocking sizes delivers performance close to ATLAS level • Remains close as array size increases • Fewer L1 and L2 cache misses than ATLAS • Similar results were observed for non-square matrices as well

15. Determining Blocking Size • Exhaustively searching for blocking sizes is expensive • Intelligent exploration of blocking sizes can find very good blocking sizes while only examining a few block sizes • Our approach: • Determine the result for block size 50 • Sample higher and lower block sizes in increments of ten until results are more than 10% from optimal • Examine all of the block sizes within five of the best found in the previous step • This approach always found the best block size in our experiments • Quicker approaches could be found at the expense of finding less ideal block sizes

16. Search time required

17. Making Adaptive Compilation General • Making adaptive compilation general will require changing how compilers work • Adaptive compilation is limited by the decisions the compiler exposes • If the MIPSpro compiler only allowed blocking to be turned on and off our experiments would not have been possible • The interface between adaptive system and compiler needs to allow complex communication • Which transformations are applied • Granularity • Optimization scope • Detailed parameter settings

18. Conclusions • Adaptively selecting the appropriate blocking size for DGEMM provides performance close to ATLAS • The standard compiler’s performance drops off for larger array sizes • Only a small portion of possible block sizes needs to be examined • Making adaptive compilation a successful technique for a wide variety of applications will require changes to the design of compilers

19. Extra slides begin here.

20. DGEMM running times for varying M

21. DGEMM running times for varying N

22. DGEMM running times for varying K