1 / 28

Constructing a Performance Database for Large-Scale Quantum Chemistry Packages

Constructing a Performance Database for Large-Scale Quantum Chemistry Packages. Meng-Shiou Wu 1 , Hirotoshi Mori 2 , Jonathan Bentz 3 , Theresa Windus 2 , Heather Netzloff 2 , Masha Sosonkina 1 , Mark S. Gordon 1,2. 1 Scalable Computing Laboratory, Ames Laboratory, US DOE

chancellor-steele
Download Presentation

Constructing a Performance Database for Large-Scale Quantum Chemistry Packages

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Constructing a Performance Database for Large-Scale Quantum Chemistry Packages Meng-Shiou Wu1, Hirotoshi Mori2, Jonathan Bentz3, Theresa Windus2, Heather Netzloff2, Masha Sosonkina1, Mark S. Gordon1,2 1Scalable Computing Laboratory, Ames Laboratory, US DOE 2Department of Chemistry, Iowa State University 3Cray Inc.

  2. Introduction • Why Quantum Chemistry (QC)?? • Applications to • Surface science • Environmental science • Biological science • Nanotechnology • Much more… • Provides a way to treat problems with greater accuracy • With advent of more powerful computational tools, QC can be applied to more and more problems

  3. Introduction:Three Major QC Software Packages • All provide various computations to calculate physical properties of molecules including energies, geometries, reaction pathways, etc. • Some calculation types are the same, but some are unique to the specific package

  4. Introduction • We have used the Common Component Architecture (CCA) framework to enable interoperability between GAMESS, NWChem and MPQC • Obtain a palette of components to utilize functionalities between each package

  5. Insight into CCA:Example--Core Interfaces for Integral Computations IntegralEvaluatorFactoryInterface integral evaluator factory CCA interface Integral Integral Integral Integral Evaluator1 Evaluator2 Evaluator3 Evaluator4 component Interface Interface Interface Interface integral integral integral integral class evaluator1 evaluator2 evaluator3 evaluator4 Chemistry Package NWChem GAMESS MPQC • IntegralEvaluatorNInterface for N-center integral , N = 1,2,3, 4. • IntegralEvaluatorFactoryInterface provides references to the integral evaluators from a chemistry program

  6. Background • Questions to ponder: • Easy selection of components with the best efficiency • Find compromises between efficiency and accuracy • Rely on Computational Quality of Service (CQoS) to address these issues: • Automatic selection and configuration of components to suit a particular computational purpose • Special requirements for scientific components: • High efficiency and parallel scalability • Functional qualities, such as the level of accuracy achieved for a particular algorithm • Core question in our CQoS research: • Need an approach for the large-scale applications

  7. Motivation for CQoS • Three basic steps for performance analysis • code instrumentation • performance evaluation • data analysis • This is straightforward IF applied to smaller packages • Must be augmented to deal with large scale applications • Need mechanisms to handle performance data

  8. Issues to be Addressed • Large number of quantum chemistry computations available in each QC packages • Individual chemists may not know all of the available computations and the algorithmic details of these computations • Construction of the database must allow/encourage participation of many chemists to try to obtain a ‘big picture’ view • The construction process should be as automated as possible • No need for chemists to select the right performance tools. • Chemists should not need to manually handle the large amounts of performance data • Management of performance data and the database should be transparent to chemists.

  9. Stages in the Development of CQoS Tuning Rules Stage 1 Stage 2 Stage 3 Stage 4 Develop/Collect Capable Implementations Performance Evaluation Data Analysis Tuning Rules Development Interoperating mechanisms Source code instrumentation Develop Analysis Procedures Performance modeling Data collecting Exploring important parameters Heuristic tuning rules Minimizing overhead Performance database Analyze the relationships between parameters Mechanisms to incorporate tuning rules

  10. Stage 1: Collect/Develop/Incorporate Implementations • Goal: • Develop, collect and integrate capable methods/implementations • For packages integrated with CCA, the goal is to develop interoperating mechanisms. • Overhead introduced by the interoperating mechanisms must be minimized Interoperating Mechanisms Minimizing Overhead

  11. Stage 2: Performance Evaluation • Goal: • Generate useful performance data and explore methods to easily manipulate large amounts of data • This is our current area of focus Source code Instrumentation Data Collection Performance Database

  12. Stage 3: Performance Analysis • Goal: • Use the collected performance data and the constructed database to conduct analysis. • Very complex process for quantum chemistry computations • Example to follow… Develop Analysis Procedures Exploring important parameters Analyze the relationships between parameters

  13. Stage 4: Tuning Rules Development • Goal: • Develop mechanisms to select one method among a pool of methods. • Depends on the results of performance analysis. • Two common approaches: • develop performance models • use heuristic tuning rules. • Tuning rules must be integrated into the original interoperating mechanisms to be useful. Performance Modeling Heuristic Tuning rules Mechanisms to Incorporate Tuning rules

  14. Performance Evaluation:Source Code Instrumentation • Insert performance evaluation functions into application source codes. • Straightforward with automatic instrumentation capability provided by existing performance tools. • Limitations exist when you need only part of the overall performance data • Expected performance data may not be generated

  15. Source Code Instrumentation(CQoS with GAMESS) • Example: • In many cases, manual instrumentationis unavoidable in GAMESS performance evaluation. A whole computation Computation 1 Computation 2 Computation 3 communications

  16. GAMESS Performance Tools Integrator(GPERFI) • Built on top of existing performance tools such as TAU (Tuning and Analysis Utilities) or PAPI • Allows more flexible instrumentation mechanisms specific to GAMESS • Provides mechanisms to extract I/O, communication, or partial profiling data from the overall wall clock time

  17. Performance Evaluation:Data Collection Discard, remove its related data from database. Uploading data to the database through PerfDMF Post processing TAU performance data. Genetate application metadata from the input file/out log files. No Decide if the input file should be tested on the other platforms Conduct performance analysis with PerfExplorer Run the testing on the experimental cluster Yes Output performance analysis results. Format verification Test the input file on different platforms. Input files uploading / results viewing interface View results and provide comments. Provide input files for experiments

  18. Performance Evaluation:Performance Database • Simply conducting many performance evaluations and putting the performance data into the database is not going to work!!! • Need to properly collect metadata that are related to performance data. • Need to build relationships between metadata. • Need participation of chemists!!!

  19. CH3 OH NH2 Performance Evaluation:Example of Construction Sequence • Chemist defines ‘molecule similarity’ between a set of molecules. • Conduct performance evaluation for the base molecule. • Use the performance data collected from the base molecule to conduct performance analysis and prediction of the other molecules with similar structure. Benzen (bz) Hydroxyl-bz Methyl-bz Amino

  20. Performance Evaluation:Performance Database • Use PerfDMF (Performance Data Management Framework), included in the TAU toolkit, for data manipulation • Use MySQL as the database engine underneath PerfDMF • Pros: Data manipulation is simplified. • Cons: Data manipulation is restricted by the schema defined by PerfDMF Performance Database PerfDMF MySQL

  21. Metadata Mapping Trial Application Experiment Experimental runs with different system settings GAMESS Computations Energy Experiment set 1 Metadata (Platform 1, CPU, cache.., etc.) Metadata (conv-SCF, .., etc) Experiment set 2 Application characteristics Metadata (Platform 2, CPU, cache.., etc.) Experiment set 3 Metadata (Platform 3, CPU, cache.., etc.) System characteristics … Energy Experiment set 1 Metadata (directSCF, .., etc) Metadata (Platform 1, CPU, cache.., etc.) Experiment set 2 Metadata (Platform 2, CPU, cache.., etc.) Experiment set 3 Metadata (Platform 3, CPU, cache.., etc.) … …

  22. Hierarchy of Quantum Chemical Calculation in GAMESS Molecule 1. Molecule $data • # of Atoms & electrons • Structure (Linearity/Planarity of the Molecule) Energy Gradient Hessian 2. Property 3. Level of Theory HF & post-HF DFT $contrl • BLYP • PBE • B3LYP etc. HF CI CC MPn 4. Basis Sets • # of basis functions • Angular momentum $basis 5. Integral Algorithm • Conventional (stored on disk) • Direct (calculated ‘on the fly’) $scf

  23. Detail of the Performance Evaluation Stage Performance Analysis Tuning Rules Development Performance Evaluation Source code Instrumentation (GPERFI for GAMESS, TAU/PDT for NWChem) Exploring important parameters Performance Modeling Data Collecting (Python programs) Heuristic Tuning Rules Analyze the relationship between parameters Application Metadata Performance Data System Metadata Performance Database PerfDMF PerfExplorer MySQL

  24. Seaborg Bassi (1) (2) On Bassi, the latency of global sum (GSUM) increase to certain ratio that’s not neglectable when we increase the number of processors used per node. On Seaborg, the ration of global sum is not very important. On Seaborg, parallel write (PWRT) is not an issue until we use 16 processors per node. On Bassi, we did not have problem with parallel write. Complexity in Performance Analysis: A Case Study(Conventional Integral Calculation) Runtime (sec) Number of nodes/Number of Proc per node

  25. What Needs to be Analyzed? • Why does the cost of parallel write (PWRT) jump dramatically when using 16 processors per node? • When does global sum (GSUM) become the performance bottleneck? • Investigation of many subroutines to study the relationships among • CPU speed • I/O bandwidth/ latency • data size • communication bandwidth/latency • number of processors used per node

  26. Conclusions • Initial work to construct a performance database that can ease the burden in manipulating performance data for quantum chemists • Data manipulation is mostly transparent to chemists. They need only to provide their knowledge of chemistry. • The infrastructure can be used on the other applications • We are working on incorporating NWChem

  27. Acknowledgements • Funding: • US DOE SciDAC initiative • Resources: • Lawrence Berkeley National Lab-NERSC

  28. Questions/Comments??

More Related