Grid Computing at PSNC

1. Grid Computing at PSNC Jarosław Nabrzyski Poznań Supercomputing and Networking Center (PSNC) and Information Sciences Institute, Poznan University of Technology email: naber@man.poznan.pl www.GridLab.org

2. Talk Outline • What is Grid Computing about? • Background - Polish Information Infrastructure • PIONIER Program • Poznan Supercomputing and Networking Center • PSNC’s Grid Research • International collaboration • IST projects • The future (6FP)

3. The Grid Problem • Flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources From “The Anatomy of the Grid: Enabling Scalable Virtual Organizations” • Enable communities (“virtual organizations”) to share geographically distributed resources as they pursue common goals -- assuming the absence of… • central location, • central control, • omniscience, • trust relationships.

4. The Grid - Not a New Idea • Late 70’s – Networked operating systems • Late 80’s – Distributed operating systems • Early 90’s – Heterogeneous computing • Mid of 90’ - metacomputing (C.C. Catlett and L. Smarr) • Transparent access to heterogeneous and geographically distributed computational resources, • Giving a user access to the required computing power • Metacomputing -> then the Grid: 1999, C. Kesselman and I. Foster

5. How are Grids Different? • Autonomy • Heterogeneity • Focus on the user • These three differences create many of the problems but also make the system much more usable than its predecessors

6. Elements of the Problem • Resource sharing • Computers, storage, sensors, networks, … • Sharing always conditional: issues of trust, policy, negotiation, payment, … • Coordinated problem solving • Beyond client-server: distributed data analysis, computation, collaboration, … • Multi-institutional virtual orgs • Community overlays on classic org structures • Large or small, static or dynamic

7. PSNC • Founded in 1993 by the State Committee for Scientific Research • One of the 5 supercomputing centres in Poland • 4 departments: APPS, HPC, NET, SERVICES • Staff:75 people

8. PSNC’s Main Activities • POZMAN - metropolitanarea network operator • Operator of the Polish National Research Network POL-34/155 (now PIONIER network) • Management of high performance computing resources, network file servers, archive systems • Development and deployment of new broadband network services • GRID computing • Resource management • Grid Information Services • Grid Application Development Toolkits

9. PSNC’s international activities • European Grid Forum (EGrid) (http://www.egrid.org/) • 1st EGrid Workshop in Poznań (April 2000) • EGrid Testbed – European Grid Forum Testbed (demo at SC’2000) • Global Grid Forum (http://www.gridforum.org/) • GGF Steering Group (J. Nabrzyski, AD) • 1st Global Grid Forum (Amsterdam 2001) – Program Chair • 5th GGF to be held in Edinburgh, July 2002

10. EGrid Testbed • Cactus worm • Dynamic grid computing • 12 sites • Presented at SC2000

11. Cactus warm • Dynamic grid computing • Remote steering and visualization • Mobile user access • 35supercomputers at 25+ sites • 10+ teraflops!

12. EGSO CROSSGRID GRIA ENACTS EUROGRID DATAGRID GRIP GRIDLAB DAMIEN DATATAG Industry / business Science EC funded Grid project space - Links with European National efforts - Links with US projects (GriPhyN, PPDG, iVDGL,…) Applications PSNC CYFRONET PSNC ICM ICM ICM PSNC Middleware & Tools • 35MЄEC-funding for above initiatives • The volume of EC projects in which Poland participates is 13MЄ Underlying Infrastructures

13. GridLab – A Grid Application Toolkit and Testbed • PSNC - Project Coordinator • 11 institutions/vendors from Europe, 3 from the US, • The total budget:5.086 Euro • 3 years project, started January 2002 • Contact: Jarek Nabrzyski – Project Manager, naber@man.poznan.pl

14. GridLab Participants • Poznań Supercomputing and Networking Center, Poland (PSNC) – Project Coordinator, • Albert Einstein Institute (AEI), Germany, • SZTAKI, Hungary • Masaryk University, Czech Republik, • University of Lecce, Italy • Konrad Zusse Centrum (ZIB), Germany • University of Wales, UK • Vrije University, Netherlands, • Sun Microsystems Gridware GmbH, Germany • Compaq EMEA, France • University of Athens, Greece • Argonne National Laboratory (Ian Foster’s group) • ISI (Carl Kesselman’s group) • University of Wisconsin (Miron Livny’s group) • Several subcontracting sites provide additional testbed resources • Many other partners will work unfunded (GGF Apps WG connections)

15. Grand Challenge Applications • Black hole simulations, • Gravitational wave detection, • Other generic applications.

16. GridLab Aims • Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community). • Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations. • Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.

17. “Grid Application Toolkit” or GAT Provides a layer between applications and emerging grid technologies. Provides an application developer orientated API, allowing the flexible use of different tools and services, as well as providing protection from developing software. “GridLab Testbed/VO” Diverse controllable environment for developing and testing applications and tools, software maintained by people who know it. Solution End Users GAT Tool Developers GAT-API Developers Grid Infrastructure Developers

18. What GridLab Isn’t … • Don’t want to develop low level Grid Infrastructure • Don’t want to repeat work which has already been done (want to incorporate and assimilate it … Globus APIs, OGSA, ASC Portal (GridSphere/Orbiter), GPDK, GridPort, DataGrid, GriPhyn)

19. Architektura GridLab

20. We see something, but too weak. Please simulate to enhance signal! NCSA Potsdam WashU 1Tbit/sec Hong Kong RZG Thessaloniki LANL LANL Now.. Grid Scenario Resource Broker: NCSA + Garching OK, but need 10Gbit/sec… OK! Resource Estimator Says need 5TB, 2TF. Where can I do this? Resource Broker: LANL is best match…

21. WP1: Grid Application Toolkit (AEI) This is a key component of GridLab - link between Grid middleware and applications, usable by any conforming application or middleware component. Requiring input from, and connecting to, most other workpackages and components. WP2: Cactus Grid Application Toolkit (AEI) Provides an extended GAT interface for Cactus, a very general toolkit framework supporting different Grid applications, from astrophysics to chemical engineering. Cactus will be one of the primary application drivers for the GAT, and the project generally. WP3: Work-flow Application Toolkit (CARDIFF) Will develop Grid capabilities for a widely used dataflow programming environment, Triana, used in gravitational wave and other data analysis areas. Workpackages

22. Workpackages (cont.) • WP4: Grid Portals (AEI) • Will be highly application driven, aimed at providing uniform, flexible and intuitive user access to Grid resources from anywhere, as well as administration tools for maintaining a Grid environment. • WP5: Testbed management (MU) • Will administrate and maintain an active development testbed across roughly a dozen EU sites (leveraging the work of the EGrid), deploying technologies as they are developed by the project. This workpackage will also coordinate with sites in the USA-based NCSA Alliance and others to test and develop interoperability. • WP6: Security (PSNC) • Will develop the required security mechanisms and will ensure the integration of all the technologies developed under other WPs, taking into account the various local security requirements and state of the art solutions.

23. Workpackages (cont.) • WP7: Adaptive Application Components (VU) • develops a set of components and APIs to be plugged into the toolkit, for example to take monitoring information and implement basic techniques for short-term forecasting and behavior adaptation/optimization. • WP8: Data Handling and Visualization (ZIB) • will provide Grid aware techniques for data management, analysis, and visualization, needed especially for applications that make use of multiple sites in a dynamic, time dependent manner, leaving data unpredictably scattered across the Grid. • WP9: Resource Management (PSNC) • will develop resource need estimators, resource brokers, and other tools, for both Grid users and the applications themselves to make intelligent decisions about which Grid resources should be used at any instant in the lifetime of a simulation.

24. Workpackages (cont.) • WP10: Information Services (ISUFI) • will extend existing Grid middleware toolkits with dynamic features needed by applications to select appropriate Grid resources and to provide simulation information to collaborative user groups. • WP11: Monitoring (SZTAKI) • will develop new components that will fit in the general Grid monitoring architecture to support application steering, adaptive monitoring, and automatic analysis and prediction of performance data. • WP12: Access for mobile users (PSNC, ZIB) • will develop and test Grid access and monitoring technologies through a variety of mobile devices,

25. Workpackages (cont.) • WP13: Information Dissemination and Exploitation (PSNC) • will ensure the active dissemination of the project results through a variety of channels, including active participation in international organizations (e.g. GGF), co-development with other Grid projects in the USA and EU, participation in international conferences, training programs, instruction of GridLab technologies into various communities, and introduction into the commercial vendor world. • WP14: Project Management (PSNC) • day-to-day scientific, financial and administrative management of the project, including carefulorchestration and monitoring of work across groups, major project decisions, liaisons with externalprojects and with the international advisory board, reporting

26. 6FP, Grids and PSNC • Europe needs the Distributed EU-wide Supercomputing Facility, • PSNC (and others) has already started to promote this idea, • EoI in preparation, • Strong co-operation with colleagues from UK, • Project proposal (70-100 MEuro) in preparation

27. Computational Needs not Met • Great scientific and engineering talent in EU • Many EU programs now beginning, with training mission, but: • No EU-wide facilities • No Collaborative Infrastructure • No HPC Training (except for do-it-yourself) • Rely on US connections • What do others do?? • In many cases, they do toy problems • EU science and engineering fall behind • Germany, UK, France are slight exceptions, but no large scale • Many countries (mainly NAS, but not only) simply cut off from modern science

28. Our Vision for the EU Facility • Science and Engineering Drivers • Expertise Centers (CFD, Astrophysics, Computational Biology, Aerospace, etc.) • Sheparding/building communities across Europe • Present science/engineering calculations require TB/Tflop/sec, but just scratching the surface! Need much more! • Enabling new science • Computational Science Drivers • Need Grid Software infrastructure • Cycles, networks, visualization centers • Partnerships with existing centers, networks, etc

29. Vision of the Facility • Distributed Nature • Pushes networking technologies through apps • Works to integrate European efforts, scientific, cultural, etc. • East and West at 100Gb/sec! • Multi-lambda backplane (GEANT?) • Unique EU facility in the HPC world • Leapfrog Teragrid with Big Machine, but also • Draw on unique, more diverse EU research strengths • Beginning of a Worldwide Grid Effort • Unique challenges to integrating diverse EU grids • Already strong support from US, NSF, DOE principals • Scientists/engineers know they have expanding facility over coming decades (can affect their projects)

30. Major Center Major Center Our Vision Satellite Satellite Satellite Satellite Satellite Satellite

31. More info / summary • www.gridlab.org • www.man.poznan.pl • www.kbn.gov.pl