How Grid Computing May Contribute to Petascale Computing

1. How Grid Computing May Contribute to Petascale Computing Marian Bubak AGH University of Science and Technology, Cracow, Poland bubak@agh.edu.pl Włodek Funika, Bartosz Baliś, Tomek Gubała, Maciej Malawski, Marcin Radecki, Katarzyna Rycerz, Tomasz Szepieniec Peter M.A. Sloot, Roland Wismueller

2. Overview • Trends in grid systems • Environment for application steering • Monitoring and performance analysis • HLA • Knowledge-based systems • Workflow construction • Semantic description • Legacy system, grid application programming • Conclusions

3. Evolution in Distributed Computing • Distributed systems operate in heterogenous environments • Large scale resource sharing • Interoperability • Communication via protocol stacks • Service oriented architectures • Open standard integration • Virtualisation of resources • Complexity of computing systems close to the limits of human capability

4. Before Grids • Parallel processing – PVM/MPI, clusters • Resource sharing systems - Condor • First metacomputing systems – Globus, Legion, Unicore • Peer2Peer systems – Seti@Home, Napster • Component frameworks – CCA • Distributed systems - CORBA

5. Grid Checklist (Ian Foster) • A Grid is a system that • Coordinates resources that are not subject to centralized control • Uses standard, open, general purpose protocols and interfaces • Delivers nontrivial qualities of services • The Grid – protocolsand interfaces are standard • GRID = Globalisation de ressources informatiques et donnees (Michel Cosnard)

6. Grid Research Projects in EU IST FP6 EU Funding: 52 MILLION Started: SUMMER 2004 GRIDCOORDBuilding the ERA in Grid research K-WF GridKnowledge basedworkflow & collaboration inteliGRIDSemantic Grid based virtual organisations Grid-based generic enabling application technologies to facilitate solution of industrial problemsSIMDAT OntoGridKnowledge Services for the semantic Grid UniGridSExtended OGSAImplementation based on UNICORE EU-driven Grid services architecture for businesS and industry NextGRID Mobile Grid architecture and services for dynamic virtual organisations Akogrimo DataminingGridDataminingtools & services HPC4UFault tolerance,dependabilityfor Grid European-wide virtual laboratory for longer term Grid research-creating the foundation for next generation Grids CoreGRID ProvenanceTrust and provenance for Grids Specific support action Integrated project Network of excellence Specific targeted research project (Thanks to Max Lemke)

7. Flood Simulation Meteo/ Pollution Particle Physics Medical Support Applications Links Plugin Plugin Performance Analysis Migrating API Desktop Performance Prediction Tools Links Protocol Plugin Portal (OMIS) SOAP MPI Verification Benchmarks SOAP SOAP Application Monitoring Roaming Infrastructure Monitoring SOAP OCM-G API Services Access Server API SOAP API API API API (JMX) Post- processing Scheduler MPI Library SOAP API API Visualization Kernel Network Monitoring DataGrid SOAP Data Access Globus Toolkit API CrossGrid Tools and Services T e s t b e d • 17 sites • 9 countries • over 200 CPUs • 4 TB of storage

8. Blood Flow Visualization Blood Flow Simulation MR Image Storage GVK LB Solver ce2 (NIKHEF) Patient in an MRI Scanner MR Image Segmentation Blood Flow Rendering in VR Portal Login and Grid Proxy Creation Virtual Node Navigation and Grid Data Transfer Bypass Placement and LB Mesh Generation Simulation Job Submission Job Monitoring Virtual Medical Support Medical Data ce (Linz) globus-lumc (Leiden) D-VRE mn (Virtual Operating Theatre at the UvA)

9. Flood Simulation Data sources Meteorological simulation Hydrological simulation Hydraulic simulation Portal

10. Application Monitoring TOOL - VISUALIZATION Standard Interface (OMIS) OCM-G – MONITORING OF APPLICATION System-specific interface RUNNING APPLICATION

11. OCM-G Monitoring System

12. Performance Analysis Tool OCM-G User Interface Measurement Interface & Visualization Interface (OMIS) Component OCM-G Monitoring System Main Service High-Level Manager Analysis Service Component ... Managers ... Local Monitors Application Performance ... Performance Modules Measurement Analysis Tool ... Component Application P1 P2 P3 Pn G-PM Processes

13. Definition of Measurements Metrics Objects Functions Partner objects Sites Nodes Processes Aggregation in time Aggregation in space

14. Definition of Visualizers • Visualization type: • Bar graph • Curve diagram • Histogram • Pie chart • Matrix diagram • Parameters: • Scales • Update interval

15. OCM-G – Features • On-line operation • Support for multi-site grid applications • Low perturbation • Techniques for data rate reduction • Lightweight and fast socket-based communication • Flexible, services-drivendesign • No fixed metrics but a set of flexible services to construct metrics with desired semantics • Enables custom metrics in G-PM • Extendible • Additional services can easily be added • Loaded dynamically at run-time • Secure • GSI-based security • Minimal security requirements • Autonomous and standardized • Standard interface • Minimized effort of porting OMIS-based tools across platforms • Enabled interoperability of multiple tools monitoring a single application.

16. LB - Interaction requires much more... Static task load Dynamic task load Static task allocation Predictable reallocation Dynamical reallocation Static resource load Dynamic resource load

17. Node A Node B PVMtask 1 PVMD A PVMD B Node C PVMtask 2 PVMD C Dynamite –PVM Initial State Two PVM tasks communicating through a network of daemons Migrate task 2 to node B

18. Node A Node B Newcontext PVMtask 1 PVMD A PVMD B Node C Program PVM Ckpt PVMD C Dynamite –Prepare for Migration Create new context for task 2 Tell PVM daemon B to expect messages for task 2 Update routing tables in daemons (first B, then A, later C)

19. Dynamite –Checkpointing Node A Node B Newcontext PVMtask 1 PVMD A PVMD B Node C Program PVM Ckpt PVMD C Send checkpoint signal to task 2 Flush connections Checkpoint task to disk

20. Dynamite –Cross-cluster checkpointing Node A Node B Helper task PVMtask 1 PVMD A PVMD B Node C Program PVM Ckpt PVMD C Send checkpoint signal to task 2 Flush connections, close files Checkpoint task to disk via helper task

21. Dynamite –Restart Execution Node A Node B NewPVM task 2 PVMtask 1 PVMD A PVMD B Node C PVMD C Restart checkpointed task 2 on node B Resume communications Re-open & re-position files

22. User Site A Application Benchmark Application Broker HLA Speaking Federate Services Monitoring Client Service Code Services RTIExec Migration Service Service Site B Broker Infrastructure Service Monitoring Migration HLA Services HLA Performance Support Interfacing Bus Decision Services Services Broker Support Service Services Application N - th Grid site supporting HLA Monitoring Main Service Manager Broker Support HLA Speaking Services Site Site C C Service RTIExec HLA Registry Registry Migration Interfacing Service Service Support Services RTIExec Services Service Grid site supporting HLA Grid HLA Management System • HLA interfacing services • HLA-Speaking Service for managing federates • RTIExec Service for managing RTIExec (coordination process in RTI) • Broker for setting up a federation and deciding about migration • Broker decision services • Registry for storinglocation of HLA-speaking services • Infrastructure Monitoring/Benchmarks for checking environment of HLA service • Application Monitoring for monitoring performance • Migration support services • Migration Servicefor performing migration

23. Collaborative Environment for Vascular Reconstruction reanalise modify bypass User A raw data analysis and segmentation Simulation Owner A, foreground data data data Visualisation And exploration In VE control 3D editing tool control data data Simulation Owner A background data control switch simulation control change simulation owner Communication bus (HLA-RTI) raw data analysis and segmentation control Visualisation And exploration In VE Simulation Owner B foreground data data data data 3D editing tool control data data Simulation Owner B background switch simulation control control change simulation owner modify bypass reanalise User B

24. G-HLAM - Results • Type of module (simulation root process or visualisation process) does not impact migration overhead • 12 MPI processes in each simulations • 52000 velocity vectors of blood flow in 3D space • GTv 3.2 and HLA RTI 1.3v5 Migration overhead Migration impact on simulation performance Type of modules: 1 migrated simulation and 25 visualisations Type of modules: 1 migrated simulation, 4 other simulations, varied number of visualisations

25. Execute workflow Construct workflow Monitor environment K-WfGrid Reuse knowledge Analyze information Capture knowledge Workflow Applications and Knowledge • Integrating services into coherent application scenarios • Enabling automatic construction and reuse of workflows with knowledge gathered during operation • Involving monitoring and knowledge acquisition services in order to provide added value for end users Technologies: service-oriented Grid architecture,software agents, ontologies, dynamicinstrumentation

26. Flow of Actions User User interaction through the Portal Guidances for the user Knowledge Web Portal Grid Workflow User Interface User Assistant Agent Grid Organizational Memory Information on available resources and their description Workflow composition and execution visualization User’s decisions in crucial points of execution Ontological store of knowledge Workflow Orchestration and Execution Automatic Application Builder Workflow Composition Tool Analysed and extracted knowledge Information about workflow execution Scheduler PerformanceAnalysis KnowledgeAssimilation Agent Grid Workflow Execution Service Information about performance of particular resources Information about resources and environment Execution of chosen Grid services Low Level Grid Middleware (WS-RF) Grid Performance Monitoringand Instrumentation Service Grid Resources

27. Stages of Workflow Construction Initial, abstract grid job Abstract Workflow with Service classes Partially concretized Workflow prior to execution Fully concretized Workflow after Successful execution

28. Monitoring and Performance Analysis • Monitoring and instrumentation service (MIS) • Performance analysis service (PAS) • Data representations and service interfaces

29. Goals Provide easy mechanisms for creation of components on distributed shared resources; Provide efficient communication mechanism both for distributed and local components; Allow flexible configuration of components and various application scenarios; Support native components, i.e. components written in non-Java programming languages and compiled for specific architecture. Solutions Distributed CCA-compliant framework Based on H2O metacomputing platform Uses RMIX for efficient communication Java and Jython scripting interface for assembling applications Work in progress to support native components using Babel MOCCA Component Framework

30. MOCCA implementation • CCA components instantiated as H2O pluglets • Each user can create own arena where components are deployed • Thanks to H2O kernel security mechanisms, multiple components may run without interfering

31. Legacy Software to Grid Services • Legacy software • Validated and optimized code / binaries • Follows traditional process based model of computation (language & system dependent) • Scientific libraries (e.g. BLAS, LINPACK) • Service oriented architecture (SOA) • Enhanced interoperability • Language independent interface (WSDL) • Execution within system neutral runtime environment (virtual machine)

32. Legacy Software to Grid Services Runtime Environment (adaptation layer) Service Requestor • Universal architecture enabling to integrate legacy software into service-oriented architecture • Novel design enablingefficiency, security, scalability, fault-tolerance, versatility • Current implementation: LGF framework which automates the process of migration of C/C++ codes to GT 3.2 • Further work: WSRF, message level security, optimizations, early process migration SOAP Legacy System SOAP

33. Programming Grid Applications • Separates the developer from ever-changing Grid resource layer • Seamlessly introduces dynamism into newly created applications • Provides unified access to resources by means of semantically described abstractions • Supports evolving and well organized library of applications used up-to-date • Allows easy reuse of already built applications

34. Collaboration PFT DB CAs SIM VIZ Data Access (OGSA-DAI) Runtime System hospital CE, WN, SE Ind. Patient Data ViroLab – Virtual Laboratory presentation Web Browser PDA Client Clinical Portal Driver PDA Driver application Rule-based System Scientific Tools Session Manager Stochastic Modeling Apps VO management, AAA, Data Access & Logging for Provenance Rule Engines Rule Derivers Rule Elucidators Biostat Apps grid middleware GLOBUS, EDG, CrossGrid, other grid resources Patient Cohort DB virtual organization Medical Knowledge Data Derived Rules DB Stanford Virology Lab DB Intermediate Experimental Data Measured Data Medical Apparatus Exp. Apparatus Immunology DB Scientific Literature Pharmacology DB computational resources data resources

35. Conclusions – Solutions from Grids • Large scale numerical simulations • Computationally demanding data analysis • Distributed computing and storage • Remote access to experimental equipments • A need for integration heterogeneous environments into one application • Collaborative problem solving • Virtual organisations

36. Web Sites • www.eu-crossgrid.org • www.kwfgrid.net • www.coregrid.net • www.virolab.eu • www.cordis.lu/ist/grids • www.gridforum.org