Phase Detection and Prediction on Real Systems for Workload-Adaptive Power Management

1. Phase Detection and Prediction on Real Systemsfor Workload-Adaptive Power Management Canturk ISCIMargaret MARTONOSI

2. Program Phases • Distinct and often-recurring regions of program behavior • How can we detect recurrent execution under real system variability? • How can we predict future phase patterns? • How can we leverage predicted phase behavior for workload-adaptive power management? • Can we do better than simple, reactive methods? Canturk Isci - Margaret Martonosi

3. Runtime Monitoring HardwarePerformanceCounters DynamicProgramFlow Application Power Estimation Phase Analysis Power Estimation Dynamic Management Real Measurements Research Overview • Monitor application execution via specific features • Classify features into phases • Detect/Predict phase behavior • Apply dynamic power management guided by phase predictions • Validate with real measurements Canturk Isci - Margaret Martonosi

4. Runtime Monitoring Runtime Monitoring HardwarePerformanceCounters DynamicProgramFlow HardwarePerformanceCounters DynamicProgramFlow Phase Analysis Phase Prediction Phase Classification Application Power Estimation Dynamic Management Real Measurements This Talk • Track memory accesses per instruction (Mem/Uop) via performance counters • Classify execution into phase patterns based on Mem/Uop rates • Predict future behavior with the Global Phase History Table (GPHT) predictor • Use phase predictions to guide dynamic voltage and frequency scaling (DVFS) Canturk Isci - Margaret Martonosi

5. Phases Mem/Uop 5 4 Phases 3 2 1 0 From Execution to Phases • Assign different Mem/Uop ranges to different phases • Higher phase number  more memory bound phase • Phase patterns expose available recurrence! • Simple phase definition  • Resilient to system variations • Invariant to dynamic power management actions 0.020 0.015 Mem/Uop Rate 0.010 0.005 0.000 2.80E+10 2.90E+10 3.00E+10 3.10E+10 3.20E+10 3.30E+10 Cycles Canturk Isci - Margaret Martonosi

6. Pt’ Pt’-1 Pt’-2 … … … … Pt’-N Pt’’ Pt’’-1 Pt’’-2 … … … … Pt’’-N Pt-1 Pt-2 … … … … Pt-N 20 15 : : : -1 Predicting Phases with the GPHT PHT Pred-n Age / Invalid PHT Tags PHT • Similar to a global history branch predictor • Implemented in OS for on-the-fly phase prediction Pt’ Pt’-1 Pt’-2 … … … … Pt’-N Pt’+1 GPHR Pt’’+1 Pt Pt-N-1 Pt’’ Pt’’-1 Pt’’-2 … … … … Pt’’-N Pt’’+1 : : : : : : : : : GPHR depth PHT entries Pt Pt : : : : : : : : : : : : : : : : : : Last observed phase from performance counters P0 P0 P0 … … … … P0 P0 GPHR depth Predicted Phase From GPHR(0) if no matching pattern From the corresponding PHT Prediction entry if matching pattern in PHT Canturk Isci - Margaret Martonosi

7. LastValue PHT:1024, GPHR:8 PHT:128, GPHR:8 PHT:64, GPHR:8 PHT:1, GPHR:8 Prediction Accuracies • Compare to reactive approaches (Last Value prediction) • GPHT performs significantly better for highly varying applications • Up to 6X and on average 2.4X misprediction improvement • Good performance down to 128 PHT entries • Converges to last value as PHT entries  1 100 90 80 70 Prediction Accuracy (%) 60 50 40 gap_ref gcc_200 gcc_166 mcf_inp apsi_ref gzip_log applu_in mgrid_in gcc_expr ammp_in gcc_scilab parser_ref equake_in wupwise_ref bzip2_source gcc_integrate bzip2_graphic bzip2_program Canturk Isci - Margaret Martonosi

8. Phase Driven Dynamic Power Management • Phase definitions  Memory boundedness  DVFS potential • Each predicted phase  Corresponding (V,f) setting • Implementation overview: Canturk Isci - Margaret Martonosi

9. ACTUAL_PHASE PRED_PHASE (GPHT) Mem/Uop (GPHT) 0.024 0.020 0.016 Mem/Uop 0.012 0.008 5 0.004 4 0.000 3 Phases 2 1 0 14 Power (Baseline) Power (GPHT) 12 10 8 6 4 2 0 2.1 BIPS (Baseline) BIPS (GPHT) 1.8 1.5 1.2 0.9 0.6 0.3 0 Complete Example • GPHT can accurately predict varying application behavior! • Significant power savings compared to baseline! Power [W] • Negligible performance degradation! BIPS 1.5E+09 2.0E+09 2.5E+09 3.0E+09 3.5E+09 4.0E+09 4.5E+09 5.0E+09 Instructions Canturk Isci - Margaret Martonosi

10. Improvement over Reactive Methods • 7% EDP improvement over reactive methods! • Comparableor less performance degradation! Canturk Isci - Margaret Martonosi

11. Conclusions • Phase characterizations help identify repetitive application behavior under real-system variability and dynamic management actions • Runtime phase predictions with the Global Phase History Table can accurately predict future application behavior • Up to 6X and on average 2.4X less mispredictions than reactive approaches • Dynamic power management guided by these phase predictions help improve system power/performance efficiency • 27% EDP improvements over baseline and 7% over reactive approaches • Presented research framework and real-system experiments can guide phase-oriented characterization and dynamic adaptation applications Canturk Isci - Margaret Martonosi

12. Thanks! Canturk Isci - Margaret Martonosi

13. EXTRAS • 1.1) Why care about phases examples • 1.2) Why care about pwr phases examples • 1.3) What are different features that prev studies looked at? • 2) Experiment setup details Canturk Isci - Margaret Martonosi

14. E1 E2 E3 E4 1.1) Why Care About Phases? • Characterizing execution regions Canturk Isci - Margaret Martonosi

15. ON OFF 1.1) Why Care About Phases? • Characterizing execution regions • Managing dynamic adaptation Canturk Isci - Margaret Martonosi

16. 1 Load Refs 0.9 0.8 Store Misses 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 3 8 13 Time [s] 1.1) Why Care About Phases? • Characterizing execution regions • Managing dynamic adaptation • Use current phase/behavior to predict future behavior Canturk Isci - Margaret Martonosi

17. Uncontrolled T Enforced T Power [W] Temp. [oC] Time [s] Time [s] Slow down! 1.2) Why Care About Power Phases? • Useful for: • Guiding power budget / temperature limit management • I.e. Montecito/Foxton Canturk Isci - Margaret Martonosi

18. 1.2) Why Care About Power Phases? • Useful for: • Guiding power budget / temperature limit management • Power/Temperature aware scheduling Power [W] [Bellosa et al. COLP’03] Time [s] Canturk Isci - Margaret Martonosi

19. 1.2) Why Care About Power Phases? • Useful for: • Guiding power budget / temperature limit management • Power/Temperature aware scheduling • Power balancing for multiprocessor systems/activity migration Power Power Task1 Task2 Swap hot task Core/μP 1 Core/μP 2 Speed up! Slow down! Canturk Isci - Margaret Martonosi

20. Older Canturk Isci - Margaret Martonosi

21. Application Power Estimation Phase Analysis Power Estimation Dynamic Management Real Measurements This Talk • Classify application execution into phases based on HW performance counters • Predict phase behavior • Apply dynamic power management guided by phase predictions • Validate with real measurements Runtime Monitoring HardwarePerformanceCounters DynamicProgramFlow Canturk Isci - Margaret Martonosi

22. Pt’ Pt’-1 Pt’-2 … … … … Pt’-N Pt’’ Pt’’-1 Pt’’-2 … … … … Pt’’-N Pt’+1 20 : : : : : : : : : : : : : : : : : : : : : : : : Pt’’+1 15 P0 P0 P0 … … … … P0 : : : : : : P0 -1 Predicting Phases with the GPHT PHT Pred-n Age / Invalid PHT Tags PHT • Similar to a global history branch predictor • Implemented in OS for on-the-fly phase prediction GPHR Pt Pt-1 Pt-2 … … … … Pt-N GPHR depth PHT entries Pt Last observed phase from performance counters GPHR depth Predicted Phase From GPHR(0) if no matching pattern From the corresponding PHT Prediction entry if matching pattern in PHT Canturk Isci - Margaret Martonosi