Prophesy: Analysis and Modeling of Parallel and Distributed Applications

1. Prophesy: Analysis and Modeling of Parallel and Distributed Applications Valerie Taylor Texas A&M University Seung-Hye Jang, Mieke Prajugo, Xingfu Wu – TAMU Ewa Deelman – ISI Juan Gilbert – Auburn University Rick Stevens – Argonne National Laboratory SPONSORS: NSF, NASA

2. Performance Modeling • Necessary for good performance • Requires significant time and effort http://prophesy.cs.tamu.edu

3. Outline • Prophesy Infrastructure • Modeling Techniques • Case Studies • Summary http://prophesy.cs.tamu.edu

4. Problem Statement • Given: • Performance models and analyses are critical • Requires significant development time • Parallel and distributed systems are complex • Goal • Efficient execution of parallel & distributed applications • Proposed Solution • Automate as much as possible • Community involvement http://prophesy.cs.tamu.edu

5. Prophesy System PROPHESY GUI Profiling & Instrument. Template Database Model Builder Performance Database Actual Execution Performance Predictor Systems Database DATA ANALYSIS DATA COLLECTION DATABASES http://prophesy.cs.tamu.edu

6. Automated Instrumentation • In-line data collection • Instrument at one of several pre-defined levels • Allow for user-specified instrumentation Profiling & Instrument. Actual Execution T=E * f; INSTRUMENTATION CODE for (I=1; I<N; I++){ V(I) = A(I) * C(I); B(I) = A(2I + 4); } INSTRUMENTATION CODE T=E * f; for (I=1; I<N; I++){ V(I) = A(I) * C(I); B(I) = A(2I + 4); } http://prophesy.cs.tamu.edu

7. Databases • Hierarchical organization • Organized into 4 areas: • Application • Executable • Run • Performance Statistics Template Database Performance Database Systems Database http://prophesy.cs.tamu.edu

8. Prophesy Database Application Performance Application Executable Run Modules Inputs Module_Info Function Performance Systems Compilers Functions Model Template Basic Unit Performance Function_Info Resource Connection Control Flow Data Structure Performance Model_Info Library http://prophesy.cs.tamu.edu

9. Data Analysis • Develop performance models • Make predictions • Performance tune codes • Identify best implementation • Identify trends Model Builder Performance Predictor http://prophesy.cs.tamu.edu

10. Automated Modeling Techniques • Utilize information in the template and system databases • Currently include three techniques • Curve fitting • Parameterization • Composition using coupling values http://prophesy.cs.tamu.edu

11. Analytical Equation (Octave: LSF) Performance Data Matrix-matrix multiply: LSF : 3 Curve Fitting: Usage Application Performance Function Performance Basic Unit Performance Model Template Data Structure Performance http://prophesy.cs.tamu.edu

12. Matrix-matrix multiplication, 16P, IBM SP http://prophesy.cs.tamu.edu

13. Analytical Equation (Octave: Parameterization) System Data: MPISR, MPIBC, ADDM Matrix-matrix multiply: Parameterization : Parameter(P, SGI Origin2000, N, ADDM, MPISR, MPIBC) Parameterization: Usage Systems Model Template Resource Connection http://prophesy.cs.tamu.edu

14. Modeling Techniques • Curve Fitting • Easy to generate the model • Very few exposed parameters • Parameterization • Requires one-time manual analysis • Exposes many parameters • Explore different system scenarios • Coupling • Builds upon previous techniques • Identify how to combine kernel models http://prophesy.cs.tamu.edu

15. Kernel Coupling • Two kernels (i & j) • Three measurements • Pi: performance of kernel i isolated • Pj: performance of kernel j isolated • Pij: performance of kernels i & j coupled • Compute Cij = Pij + Pi Pj http://prophesy.cs.tamu.edu

16. Coupling Categories • Cij = 1: no coupling • Cij > 1: destructive coupling • Cij < 1: constructive coupling http://prophesy.cs.tamu.edu

17. Coupling Categories Cij = 1: No Coupling Kernel A Kernel A Kernel B Shared Resource Kernel B Cij > 1: Destructive Coupling Cij < 1: Constructive Coupling Kernel B Kernel A Kernel A Kernel B Shared Resource Shared Resource http://prophesy.cs.tamu.edu

18. = (CAB * PAB + CAC * PAC ) PAB + PAC = (CAB * PAB + CBC * PBC ) PAB + PBC = (CBC * PBC+ CAC * PAC ) PBC + PAC Using Coupling Parameters • Use weighted averages to determine how to combine coupling values • Example: • Given the pair-wise coupling values Want: T = EA + EB + EC Kernel A Kernel B Kernel C http://prophesy.cs.tamu.edu

19. = 0.8472 = 0.7591 = 0.8407 Composition Method • Synthetic kernels (array updates) Kernel A (196.44) Kernel B (207.16) Kernel C (574.19) Actual total time: 799.63s Coupling time: 776.52s (Error: 2.89%) Adding individual times: 971.81s (Error: 23%) http://prophesy.cs.tamu.edu

20. Analytical Equation (Octave: Coupling) Coupling Values and Performance data Data and System Info Adjacent Kernels Coupling Method: Usage Run Functions Inputs Function Performance Systems Control Flow Coupling http://prophesy.cs.tamu.edu

21. Case Studies • Predication: Resource Allocation • Grid Physics Network (GriPhyN) • Utilizes Grid 2003 infrastructure • GeoLIGO application • Prediction: Resource Allocation • AADMLSS: Educational Application • Utilizes multiple servers http://prophesy.cs.tamu.edu

22. Case 1: GEO LIGO (GriPhyN) • The pulsar search is a process of finding celestial objects that may emit gravitational waves • GEO (German-English Observatory) LIGO (Laser Interferometer Gravitational-wave Observatory) pulsar search is the most frequent coherent search method that generates F-statistic for known pulsars http://prophesy.cs.tamu.edu

23. GRID 2003 GriPhyN Resource Selection Chimera Virtual Data System Prophesy Transform using VDL Grid Middleware Ganglia Submission Monitoring http://prophesy.cs.tamu.edu

24. Prophesy Interface Predictor Resource Selector Application Name Rankings of sites Input Parameters, List of available sites Weights of each site http://prophesy.cs.tamu.edu

25. Grid2003 Testbed http://prophesy.cs.tamu.edu

26. Execution Environment http://prophesy.cs.tamu.edu

27. Experimental Results

28. Case Study 2: AADMLSS African American Distributed Learning System (AADMLSS) developed by Dr. Juan E. Gilbert http://prophesy.cs.tamu.edu

29. Measure Server Performance Measure Network Performance User logs into AADMLSS YES NO Valid Username and Password? First time access? YES NO Get last concept Get default concept Display Concept Select server with best overall site performance NO Pass Quiz? NO YES Current concept Next concept (different instructor) (same instructor) Exit? YES User logs out Site Selection Process http://prophesy.cs.tamu.edu

30. Testbed Overview http://prophesy.cs.tamu.edu

31. 4-Servers http://prophesy.cs.tamu.edu

32. Results - 4 Servers http://prophesy.cs.tamu.edu

33. Results – 3 Servers http://prophesy.cs.tamu.edu

34. Results – 3 Servers http://prophesy.cs.tamu.edu

35. Results – 2 Servers http://prophesy.cs.tamu.edu

36. Summary • Prophesy • Two case studies with resource allocation • Geo LIGO: on average 33% better than load-based selection • AADMLSS: on average 4-11% better than load-based selection • Future work • Continue extending application base • Work on queue wait time predictions http://prophesy.cs.tamu.edu

37. Performance Analysis Projects • Prophesy • http://prophesy.cs.tamu.edu • Published over 20 conference and journal papers • PAPI • http://icl.cs.utk.edu/papi/ • SCALEA-G • http://www.dps.uibk.ac.at/projects/scaleag/ • PerfTrack • http://web.cecs.pdx.edu/~karavan/perftrack • Paradyn • http://www.cs.wisc.edu/~paradyn/ • Network Weather Service • http://nws.cs.ucsb.edu http://prophesy.cs.tamu.edu