1 / 27

Open Problems of Petascale Molecular-Dynamics Simulations

Supercomputing Applications in Science and Industry Sept. 20-21, Sunny Beach, Bulgaria. Open Problems of Petascale Molecular-Dynamics Simulations. D. Grancharov , E. Lilkova , N. Ilieva , P. Petkov and L. Litov. University of Sofia “St. Kl . Ohridski ”, Faculty of Physics.

conroy
Download Presentation

Open Problems of Petascale Molecular-Dynamics Simulations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Supercomputing Applications in Science and Industry Sept. 20-21, Sunny Beach, Bulgaria Open Problems of PetascaleMolecular-Dynamics Simulations D. Grancharov, E. Lilkova, N. Ilieva, P. Petkov and L. Litov University of Sofia “St. Kl. Ohridski”, Faculty of Physics Institute for Nuclear Research and Nuclear Energy – BAS

  2. Content • Introduction • Molecular dynamics in brief • ODE integrators • Scalability of the MD-packages GROMACS and NAMD in simulations of large systems • Workload distribution on the computing cores in the MD simulations • pp:pme ratio optimization • Outlook PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  3. Bond strength Bond angle Torsion Van der Waals interac-tions Coulomb interaction Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Method for investigation of time evolution of atomic and molecular systems • Classical description of the systems; • Empirical parametrisation of the interaction potential between atoms and molecules – molecular force field; • The force field is conservative, depending on atoms positions only, pair-aditive (NB: cut-offs, boundary conditions). PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  4. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook  MD CM: Newton equation QM: Schrödinger equation Probability to find the system at (r, t) PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  5. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Hamiltonian nature of the investigated dynamics Case of quadratic kinetic energy If, in addition, separable Flow h  symplectic transformation (Theorem, Poincaré, 1899) PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  6. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook preserving the symplectic form Symplectic, i.e.: preserving oriented areas in phase space Most of the usual numerical methods (primitive Euler, classical Runge-Kutta) are not symplectic integrators Ex.: Encke/Störmer/leap frog/Verlet: • resonances •  most rapid vibrational mode • implicit IA: nonlinear; still resonance Extensions: Fixed step size  variable step size Single-step  multiple-step, multirate PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  7. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Composition methods Splitting methods Ex.Symplectic Euler & Störmer-Verlet schemes PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  8. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Combining exact and numerical flows Splitting in more than two vector fields Integrators based on generating functions Variational integrators Discretizing the action integral PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  9. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Integration algorithms with variable time step • improved performance • degradation of accuracy (trade-off vs. structure preserving) • accurate trajectories  high-order methods, small timesteps • high-order {} structural properties (E, P) • deficiency in long-term performance • complicated, unstable, chaotic traj’s: asm uch structure as possible • loss of symplecticness • simple variable / symmetrized time step • different ’s in different phase-space regions • (computational cost) Multirate methods (processes  subsystems of the ODE system) e.g. 2-, 3-, and 4-body interactions PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  10. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook point in the phase space of the system и do not commute Hamilton’s equations Liouville operator in Cartesian coordinates i = { , H} PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  11. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook One-step propagators; step t; apply on PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  12. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Second-order algorithm Fi“softer” than Fi+1 Ex.: Overall step 0, but effectively till the i-th itteration, if PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  13. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook MD-simulation performance for large systems (105 atoms and more):  scalability,  distribution of the computational load  its dependence on the functional assignment to the individual processors epidermic growth factor ~5 x 105 atoms satellite of the tobacco mosaic virus ~106 atoms E.Coli ribosome in water ~2,2 x 105 atoms PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  14. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook • Compiled with the XL compilers of IBM for the architecture of the computing nodes of BlueGene/P; • NAMD – compresed input data; • Peculiarity in the way the data is loaded into the RAM memory of the computing cores of IBM BlueGene/P: GROMACS up to 700000 atoms PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  15. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Performance: the simulation time to be obtained for 24 hours with integration step of 2fs Speed-up: the performance at 512 computing cores as reference value PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  16. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Distribution of the system parts among the computing cores:  particle decomposition (~ N x N/2)  domain decomposition long-range interactions: PME algorithm SCALASCA profiling tool: guides the optimization of parallel programs by measuring and analyzing their behavior during the run  instrumentation of the code  starting the instrumented code  data analysis (Cube 3) estimation of the efficiency, speed and parallelization behavior of the algorithms in use PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  17. в в Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook 103079 atoms 10000 steps x 2 fs = 20 ps pbc; Berendsen thermostat no LINCS or P-LINCS used Distribution of the communications:(а) interleave; (b) pp_pme; (c) Cartesian (red – higher intensity, yellow – lower intensity). PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  18. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performancepp:pme optimizationOutlook • Up to 2048 cores – similar performance • On 4096 cores the default mode is the slowest one PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  19. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook • Test system of ~ 465000 atoms • 200 steps • 1/8 of all cores – pme cores • 512 and 1024 cores (51 GB output data on 1024 cores) • total time 3.10 6 s • execution time 2,3.10 6 s • t/core (average) 4668 s • (pme 5888 s; pp 4494 s) • ~ 70 % do_md • ~ 30 % long-range electrostatics & • domain decomposition on the cores • communications 3,18.10 6 • do_md ~ 58 % • long-range el. ~ 21 % • initialization of envir. ~ 20 % • pme 10453 & pp  4174 512 cores PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  20. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook • total time 3.10 6 s • execution time 2,3.10 6 s • t/core (average) 4912 s • (pme 6829 s; pp 4637 s) • ~ 66,6 % do_md • ~ 30 % long-range electrostatic & • domain decomposition on the cores • communications 7,1.10 6 • do_md ~ 60 % • long-range el. ~ 22 % • initialization of envir. ~ 18 % • pme 13600 & pp  6100 1024 cores PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  21. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook • Test system of ~ 200000 atoms • 2000 steps • pme:pp 1:1 to 1:3 (16  8 out of 32 cores) • g_tune_pme • cut-off radius 0.9 nm  1.15 nm  strong case-dependence of the most appropriate parameter set PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  22. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook • The increasing size and complexity of the investigated objects press strongly on the reconsideration of the existing algorithms not only because of the exploding computation volumes but also because of the poor scalability with the number of the processors employed; • The performed investigations allow to clearly identify the main reasons for the increase of communication between the computing cores and thus for damping down the scalability of the code; • The multiple-time step symplectic integration algorithm we work on aims at resolving this problem. PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  23. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook Thank you for your attention! PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  24. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook • T.F. Miller III, M. Eleftheriou, P. Pattnaik, A. Ndirango, D. Newns, and G. J. Martyna, J. Chem. Phys. 116 (2002) 8649. • S. Nosé, J. Phys. Soc. Jpn. 70 (2001) 75. • R. Skeel, J.J. Biesiadecki, Ann. Num. Math.1 (1994) 1–9. • D. Janezic and M. Praprotnik, J. Chem. Inf. Comput. Sci. 43 (2003) 1922–1927. • M. Tao, H. Owhadi, J.E. Marsden, Symplectic, linearly-implicit and stable integrators with applications to fast symplectic simulatons of constrained dynamics, e-Print arXiv: 1103.4645 (2011). • Wei He and Sanjay Govindjee, Application of a SEM Preserving Integrator to MolecularDynamics, Rep. No. UCB/SEMM-2009/01, Jan 2009, Univ. of California, Berkley, 27 pp. • E. Hairer, C. Lubich, and G. Wanner, Geometric Numerical Integration:Structure-Preserving Algorithms for Ordinary Differential Equations. (Springer, Heidelberg, Germany, second ed., 2004). PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  25. Molecular DynamicsODE IntegratorsScalability of GROMACS and NAMD (above 2048 cores) Workload distribution and dd_order performance pp:pme optimizationOutlook  R.S. Herbst, Int. J. Radiat. Oncol. Biol. Phys. 59 (2 Suppl) (2004) 21–6:  H. Zhang, A. Berezov, Q. Wang, G. Zhang, J. Drebin, R. Murali, M.I. Greene, J. Clin. Invest.117/8 (2007) 2051-2058.  F. Walker, L. Abramowitz, D. Benabderrahmane, X. Duval, V.R. Descatoire, D. Hénin, T.R.S. Lehy, T. Aparicio, Human Pathology 40/11 (2009) 1517–1527.  http://www.ks.uiuc.edu/Research/STMV/  E. Villa et al., Proc. Natl. Acad. Sci. USA 106 (2009) 1063–1068.  K. Y. Sanbonmatsu and C.-S. Tung1. Journal of Physics: Conference Series 46 (2006) 334–342. PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  26. Spare slides PRACE Regional Conference “Supercomputing Applications in Science and Industry”

  27. Algorithms for solving the equation of motion Error at every step Accumulated error Computer aided drug design

More Related