Introduction of Grid Computing

1. Introduction of Grid Computing ASNET-AM Annual Report,Yerevan, Armenia, 17 December 2008 Hrachya Astsatryan Institute for Informatics and Automation Problems National Academy of Sciences of the Republic of Armenia hrach@sci.am The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no. 211338

2. Outline Definition of Grid computing Grid Computing Components E-Infrastructures Grid Monitoring & Information Services

3. Grid Definition: Word Meaning • The term Grid computing or Grid suggest a computing paradigm similar to an electric power grid - a variety of resources contribute power into a shared "pool" for many consumers to access on an as-needed basis.

4. Grid Definition (2) In 1998 Ian Foster and Carl Kesselman (The Grid: Blueprint for a New Computing Infrastructure) • “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” In 2002 Ian Foster (What is the Grid? A Three-Point Checklist) A Grid is a system that • coordinates resources that are not subject to centralized control • using standard, open, general-purpose protocols and interfaces • to deliver nontrivial qualities of service.”

5. Grid Definition (3): A Working Definition • A distributed computing environment that coordinates • Computational jobs • Data placement • Information management • Scales from one computer to thousands • Capable of working across many administrative domains

6. Outline Definition of Grid computing Grid Computing Components E-Infrastructures Grid Monitoring & Information Services

7. Grid Computing Components • Distributed People • Research communities who need to share data, or codes, or computers, or equipment to work on and understand common problems • Example: Astrophysics Network: relativists, astrophysicists, computer scientists, mathematicians, experimentalists, data analysts. • Distributed Resources • Computers: supercomputers, clusters, workstations • Storage devices, databases, networks • Experimental equipment: telescopes/interferometers

8. Grid Computing Components (2) • Software infrastructure • Links all these together • Low level: security, information, communication, … • Middleware: data management, resource brokers, web portals, monitoring, workflow, … • Examples • Globus • Condor • Glite

9. Grid Computing Components (3)Virtual Organizations • Groups of organizations that use the Grid to share resources for specific purposes • Support a single community • Deploy compatible technology and agree on working policies • Security policies – difficult • Deploy different network accessible services: • Grid Information • Grid Resource Brokering • Grid Monitoring • Grid Accounting

10. Grid Computing Components (10) Access M I D D L E W A R E Suprecomputers, clusters User Experiments, sensors, etc.. Visualization Internet, networks

11. Hardware Components: Brief History of Computing • 1980: "DOS addresses only 1 Megabyte of RAM because we cannot imagine any applications needing more." -Microsoft on the development of DOS. • 1981: "640k ought to be enough for anybody." -Bill Gates

12. Hardware Components (2):Basic Elements • Distributed systems built from • Computing elements (processors) • Communication elements (networks) • Storage elements (disk, attached or networked) • New elements • Visualization/interactive devices • Experimental and operational devices

13. Hardware Components (4):Basic Elements Supercomputers • Definition of supercomputer • Machine on Top500.org? • Machine costing over $1M? • Most powerful machines • One-of-a-kind • Top 1 (Latest 2008) • Roadrunner - BladeCenter QS22 (US) 1026TF • Top 1 (November 2006) • IBM Blue Gene/L (US) 131k procs, 280 TF • Top 1 (2003) • Earth Simulator (JAPAN) 5K procs/36 TF (6)

14. Outline Definition of Grid computing Grid Computing Components E-Infrastructures Grid Monitoring & Information Services

15. E-Infrastructures: network layer concertation policies support GO4IT MUPBED PORTA OPTICA STUDY EU-QoS LOBSTER GEANT geo. extension user involv. SEEREN2 AUGERACCESS ORIENT TEIN2 EXPRESS EUMEDIS ALICE

16. E-Infrastructures (2): Grid layer concertation policies support ISSEG ICEAGE ETICS data EGEE grid geo. extension user involv. GRIDCC BALTICGRID BIOINFOGRID HEALTHeCHILD EUCHINAGRID SEEGRID applications DILIGENT EUMEDGRID EELA

17. E-Infrastructures (3): Data layer concertation policies DATA Standards OGF data grid

18. E-Infrastructures (4): Global perspective

19. E-Infrastructures (5): Grid Examples Potential for linking ~80 countries by 2008

20. E-Infrastructures (6): Collaborating Projects Infrastructures geographical or thematic coverage Support Actions key complementary functions Applications improved services for academia, industry and the public

21. SEE GRID SCI Project Contractors GRNET Greece CERN Switzerland SZTAKI Hungary IPP-BAS Bulgaria ICI Romania TUBITAK Turkey ASA/INIMA Albania UoBL Bosnia-Herzegovina UKIM FYR of Macedonia UOB Serbia UoM Montenegro RENAM Moldova RBI Croatia IIAP-NAS-RAArmenia GRENA Georgia Third Party ssociate universities / research centres The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no. 211338

22. SEE GRID SCI: converged communication and service infrastructure for SEE The SEE-GRID-SCI initiative is co-funded by the European Commission under the FP7 Research Infrastructures contract no. 211338

23. SEE GRID SCI Open Applications: Earthquake

24. SEE GRID SCI Open Applications: Meteorology Advances in numerical weather prediction (NWP) has been always very closely related with advances in computing sciences as NWP requires numerical calculations that are also parallelizable. The computer resources needed for NWP applications are important both in terms of CPU usage and disk storage. Although many institutions are working/ have experience on NWP, they may not have access to the necessary computer resources for operational implementation of such applications or for large experiments. So the porting of any NWP application to the grid is a natural choice.

25. SEE GRID SCI Open Applications: Meteorology • The REFS application will allow the meteorological entities participating in the project to assess the probability of a particular weather event to occur and to provide this information to the authorities, the general public, etc, in order to help them make the necessary decisions based on this probabilistic information. • The WRF-ARW application will permit the entities participating in the project to improve the quality of the forecasts of the airflow over regions characterised by complex terrain with a positive impact to related applications such as air-pollution dispersion modelling.

26. SEE GRID SCI Open Applications: Environment • The aim of the Monte Carlo Sensitivity Analysis for Environmental Systems application is to develop an efficient Grid implementation of a Monte Carlo technique for sensitivity studies in the domains of Environmental modeling and Environmental security. The developed application will be applied for studying the damaging effects that can be caused by high pollution levels (especially effects on human health), when the main tool will be the Danish Eulerian Model (DEM). • Multi-Scale Atmospheric Composition Modelling.Atmospheric composition directly affect many aspects of life. AQ studies are fundamental for the future orientation of national, regional and Europe’s Sustainable Development strategy.Expected results and their consequences • high quality scientifically robust assessments of the air pollution and its origin from urban to local to regional (Balkan) scales • Determination of the main pathways and processes that lead to atmospheric composition formation in different scales

27. Armenian National Grid Initiative • The Armenian National Grid Initiative (ArmNGI) represents an effort to establish a sustainable grid infrastructure in Armenia. The establishment of ArmNGI foundation is in process. Main aims of the initiative are; • create a national GRID development policy • to build up the national grid infrastructure • to expand the high performance computing resources with collaboration of academic and commercial participants • to give the information to the national user community about high performance computing, grid infrastructure and international grid projects • to improve national applications • to take place the international grid projects actively

28. Armenian National Grid Initiative • State Scientific Committee of the Ministry of Education and Science of the Republic of Armenia • National Academy of Sciences of the Republic of Armenia • State Engineering University of Armenia • Yerevan State University • Yerevan Physics Institute after A. Alikhanian • Institute for Informatics and Automation Problems of the National Academy of Sciences of the Republic of Armenia • Armenian e-Science Foundation

29. European Commission “…for Grids we would like to see the move towards long-term sustainable initiativesless dependent upon EU-funded project cycles” Viviane Reding, Commissioner, European Commission, at the EGEE’06 Conference, September 25, 2006

30. European Grid Initiative Goal: • Creating a long-term sustainability of grid infrastructures in Europe Approach: • Establishment of a new federated model bringing together National Grid Initiatives (NGIs) to build the EGI Organisation

31. Characteristics of NGIs Each NGI • … should be a recognized national body with a single point-of-contact • … should mobilise national funding and resources • … should operate the national e-Infrastructure • … should support user communities (application independent, and open to new user communities and resource providers) • … should contribute and adhere to international standards and policies Responsibilities between NGIs and EGI are split to be federated and complementary

32. 38 National Grid Initiatives www.eu-egi.org

33. European Grid Initiative EGI Organisation: • Coordination and operation of a common multi-national, multi-disciplinary Grid infrastructure • To enable and support international Grid-based collaboration • To provide support and added value to NGIs • To liaise with corresponding infrastructures outside Europe

34. Outline Definition of Grid computing Grid Computing Components E-Infrastructures Grid Monitoring & Information Services

35. Grid Monitoring Monitoring provides information for several purposes • Operation of Grid • Monitoring and testing Grid • Deployment of applications • What resources are available to me? (Resource discovery) • What is the state of the grid? (Resource selection) • How to optimize resource use? (Application configuration and adaptation) • Information for other Grid Services to use

36. Grid Monitoring (2) Monitoring information is either static or dynamic, broadly. • Static information about a site: • Number of worker nodes, processors • Storage capacities • Architecture and Operating systems • Dynamic information about a site • Number of jobs running on each site • CPU utilization of different worker nodes • Overall site “availability” • Time-varying information is critical for scheduling of grid jobs • More accurate info costs more: it’s a tradeoff.

37. Grid Monitoring (3) MonALISA http://monalisa.caltech.edu/

38. Grid Monitoring (4): GSTAT http://monalisa.caltech.edu/

39. Grid Monitoring (7): Monitoring Grid Resources • Status of resource on grid • Up/down? • How much load? Discovery • Start with a task to perform on the grid • For example, want to perform run a simulation • How do we find resources to use? • How do we choose which resource to use?