Dynamic Instrumentation of Large-Scale MPI and OpenMP Applications

1. Dynamic Instrumentation of Large-Scale MPI and OpenMP Applications Christian Thiffault, Michael Voss, Steven T. Healey, Seon Wook Kim Presented By: Jaydeep Marathe NC State University

2. The Basic Idea .. • Instrumentation needed to collect collect performance data. • Static (compile or link time) instrumentation • Dynamic (run-time) instrumentation • Feasibility study for dynamic instrumentation and dynamically • controlled static instrumentation. • Paper asserts: “Dynamic methods better than static instrumentation with • respect to execution overhead & trace data size” But that’s still debatable, as we shall see ... NC State University

3. Base Trampoline Mini-Trampoline Save Context Instrumentation Code Pre-Instrument Log_Event(…); Start_Timer(); ….. …... Restore Context Relocated Instruction Save Context Post-Instrument Restore Context Dynamic Instrumentation (with DPCL) Executing Program JMP _Test Probe Point NC State University

4. List of Active Points Yes ! Instrumentation Code Log_Event(…); Start_Timer(); ….. …... Dynamic Control of Static Instrumentation Instrumentation Library Compiler-Instrumented Program Conf_sync() { return; } …………... …………... Conf_sync() …………... Read Active List …………... …………... Point Active ? Conf_sync() …………... …………... …………... Conf_sync() …………... Instrument compile-time, decide activation at run-time. NC State University

5. Instrumentation Tool : Dynprof • For mixed (?) OpenMP + MPI programs GuideTrace OpenMP Library KAI Guide Compiler Application Source Dynprof VampirTrace Library Executable MPI Library • Applicable only to the KAI OpenMP compiler • Instrumentation : VampirTrace call to log trace Trace Data NC State University

6. Experiments .. • Compare execution times for static vs. dynamic instrumentation. • 4 ASCI benchmarks - Smg98, Sppm, Sweep3d (MPI), Umt98 (OpenMP) • IBM Power3 144-node system, each node has 8 Power3 processors. • Measure execution times with 5 scenarios: • FULL: Full Static Instrumentation +Trace logging • FULL-OFF: Full Static Instrumentation, no trace logging • SUBSET: Full Static Instrumentation, only selected functions active. • None: No instrumentation (Baseline) • Dynamic: Dynamically inserted instrumentation. NC State University

7. Results: Smg98 NC State University

8. Results: Smg98 (contd) Static approach (SUBSET), more expensive than dynamic insertion (DYNAMIC). WHY ?? - No explanation in paper. NC State University

9. Results: Smg98  Another interpretation Observe values for FULL, FULL-OFF,SUBSET. Very Close !! Compiler instrumentation inefficiency  skews results !  dynamic approach looks better. NC State University

10. Time to create and instrument • Significant cost, but only occurs at startup. • May significantly perturb application, when instrumented during run-time. NC State University

11. Summary and Conclusions • Tool to instrument MPI and OpenMP programs (Dynprof) • Trace logging/visualization through existing VampirTrace library. • Compare costs of static,dynamic and dynamically controlled static • instrumentation. • Asserts that dynamically inserted instrumentation is less expensive, as compared to static instrumentation. NC State University