Computação Grid IFSC, Julho 2005

1. Computação GridIFSC, Julho 2005 Sergio Takeo Kofuji, Prof. Dr. EPUSP

2. Motivação Sergio Takeo Kofuji

3. Computational simulation : “a means of scientific discovery that employs a computer system to simulate a physical system according to laws derived from theory and experiment” Three pillars of scientific understanding • Theory • Experiment • Simulation “theoretical experiments”

4. Can simulation produce more than “insight”? “The purpose of computing is insight, not numbers.” — R. W. Hamming (1961) “The computer literally is providing a new window through which we can observe the natural world in exquisite detail.” — J. S. Langer (1998) “What changed were simulations that showed that the new ITER design will, in fact, be capable of achieving and sustaining burning plasma.” — R. L. Orbach (2003, in Congressional testimony about why the U.S. is rejoining the International Thermonuclear Energy Reactor (ITER) consortium)

5. Can simulation lead to scientific discovery? Instantaneous flame front imaged by density of inert marker Instantaneous flame front imaged by fuel concentration Images c/o R. Cheng (left), J. Bell (right), LBNL, and NERSC 2003 SIAM/ACM Prize in CS&E (J. Bell & P. Colella)

6. Engineeringcrash testing aerodynamics Lasers & Energycombustion ICF Biology drug design genomics The imperative of simulation Applied Physics radiation transport supernovae Environment global climate contaminant transport Scientific Simulation In these, and many other areas, simulation is an important complement to experiment.

7. Engineeringcrash testing aerodynamics Lasers & Energycombustion ICF Biology drug design genomics The imperative of simulation Applied Physics radiation transport supernovae Environment global climate contaminant transport Experiments controversial Scientific Simulation In these, and many other areas, simulation is an important complement to experiment.

8. Engineeringcrash testing aerodynamics Lasers & Energycombustion ICF Biology drug design genomics The imperative of simulation Applied Physics radiation transport supernovae Experiments dangerous Environment global climate contaminant transport Experiments controversial Scientific Simulation In these, and many other areas, simulation is an important complement to experiment.

9. Engineering crash testing aerodynamics Lasers & Energycombustion ICF Biology drug design genomics The imperative of simulation Experiments prohibited or impossible Applied Physics radiation transport supernovae Experiments dangerous Environment global climate contaminant transport Experiments controversial Scientific Simulation In these, and many other areas, simulation is an important complement to experiment.

10. Engineeringcrash testingaerodynamics Lasers & Energycombustion ICF Biology drug design genomics The imperative of simulation Experiments prohibited or impossible Applied Physics radiation transport supernovae Experiments dangerous Experiments difficult to instrument Environment global climate contaminant transport Experiments controversial Scientific Simulation In these, and many other areas, simulation is an important complement to experiment.

11. Engineeringcrash testingaerodynamics Lasers & Energycombustion ICF Biology drug design genomics ITER: $5B The imperative of simulation Experiments prohibited or impossible Applied Physics radiation transport supernovae Experiments dangerous Experiments difficult to instrument Environment global climate contaminant transport Experiments controversial Experiments expensive Scientific Simulation In these, and many other areas, simulation is an important complement to experiment.

12. What would scientists do with 100-1000x? Example: predicting future climates • Resolution • refine horizontal from 160 to 40 km • refine vertical from 105 to 15km • New physics • atmospheric chemistry • carbon cycle (currently, carbon release is external driver) • dynamic terrestrial vegetation (nitrogen and sulfur cycles and land-use and land-cover changes) • Improved representation of subgrid processes • clouds • atmospheric radiative transfer

13. What would we do with 100-1000x more? Example: predict future climates Resolution of Kuroshio Current:Simulations at various resolutions have demonstrated that, because equatorial meso-scale eddies have diameters ~10-200 km, the grid spacing must be < 10 km to adequately resolve the eddy spectrum. This is illustrated in four images of the sea-surface temperature. Figure (a) shows a snapshot from satellite observations, while the three other figures are snapshots from simulations at resolutions of (b) 2, (c) 0.28, and (d) 0.1.

14. What would scientists do with 100-1000x? Example: lattice QCD • Currently available: 1 Tflop/s • Resources at the 100-200 Tflop/s level will: • enable precise calculation of electromagneticform factors characterizing the distribution of charge and current in the nucleon • make possible calculation of the quark structure of the nucleon • enable calculationof transitions to excited nucleon states • Pflop/s resources would: • enable study of the gluon structure of the nucleon, in additionto its quark structure • allow precision calculation of the spectroscopy of stronglyinteracting particles with unconventional quantum numbers, guiding experimental searches forstates with novel quark and gluon structure.

15. What would we do with 100-1000x more? Example: probe the structure of particles Constraints on the Standard Model parameters r and h. For the Standard Model to be correct, they must be restricted to the region of overlap of the solidly colored bands. The figure on the left shows the constraints as they exist today. The figure on the right shows the constraints as they would exist with no improvement in the experimental errors, but with lattice gauge theory uncertainties reduced to 3%.  

16. Workflow

17. Workflowa.k..a. The Scientific Method (in the Age of the Age of High-Speed Networks, Fast Processors, Mass Storage, and Miniature Devices) IIC contact: Matt Welsh, FAS

18. Session Server XGSP-based Control Media Servers Filters NaradaBrokering All Messaging Admire SIP H323 Access Grid Native XGSP GlobalMMCS Web Service MCU Architecture Web Services: Session Control Audio Mixer Video Mixer Codec Conversion Helix Real Streaming PDA Conversion H323/SIP Gateways Thumbnails Plus NaradaBrokering Message servers and routers Release May 15 As independent can replicate as needed Example of stream handling Needs a Grid Farm Use Multiple Media servers to scale to many codecs and many versions of audio/video mixing WebServices NB Scales asdistributed High Performance (RTP)and XML/SOAP and .. Gateways convert to uniform XGSP Messaging NaradaBrokering

19. Integração de PDA, Cell phone e Desktop Grid Access

20. GRID • Computação Técnica e Científica • Processamento, Dados, Visualisação, Instrumentação • Computadores de Alto Desempenho, Redes de Alta Velocidade • Acesso Restrito • Usuários: Especialistas • Computação Comercial

21. Introdução Sergio Takeo Kofuji

22. Níveis de Grid • Único Domínio Administrativo • Departmental • Campus • Corporativo • Regional • Estadual • Nacional • Global

23. GRIDs de Aplicações • Grid Médico & Saúde • Biblioteca Digital Multimídia • Grid de Sensores • Grid de Computação Pervasiva • Grid de Física, Biotecnologia, Ambiental • Grid de Colaboração • Media (Film…) Production and Distribution Grid • Rastreamento de Objetos/Animais/Pessoas • Jogos

24. GRID - Tipos • Processamento – Supercomputador Virtual Supercomputer; Cluster Virtual etc • Dados/Armazenamento (Storage ) • Intrumentação e Sensor • Visualisação • Colaboração

25. GRID – Componentes Físicos • Equipmentos • Vector/Parallel Supercomputer (NEC SX7, CRAY) • MPPs • Clusters • (SMP) Servers • Desktops • Notebooks • Palmtops • Cell Phones • Sensors and Actuators • RFIDs

26. GRID - Tendências • Pervasivo • Heterogêneo • Dinâmico • Orientado a Serviços

27. The GRIDs Environment User Appliance Plug-in Personal Communication Personal Shopping Hobbies, family activities Business Communication Business Dealing Business Information PC Palmtop Mobile.. Acesso Pervasive ao Grid ‘The Wall’

28. 1a. Geração Computationally intensive, file access/transfer Bag of various heterogeneous protocols & toolkits Recognizes internet, ignores web Academic Team 2a. Geração Data intensive Knowledge intensive Service based architecture Recognizes Web and Web services Global Grid Forum Industry participation Grids - Evolução

29. Typical Application Services Domain Specific Functionality Workflow Algorithms etc Visualization Typical Grid Middleware Management Identification/Scheduling Heterogeneous data management Web Services Technologies Typical Platform Instruments Data Computers Storage Network Grid - Arquitetura Simplificada

30. Managed shared virtual systems Computer science research Open Grid Services Arch Web services, etc. Real standards Multiple implementations Globus Toolkit Internet standards Defacto standard Single implementation Padrões Abertos de Grid Increased functionality, standardization Custom solutions 1990 1995 2000 2005 2010

31. GT1 GT2 OGSI Started far apart in apps & tech Have been converging WSRF WSDL 2, WSDM WSDL, WS-* HTTP Serviços Grid e Web - Convergência Grid Web WSRF - Comunidades de Grid e Web podem se mover para uma base comum

32. From Savas Parastatidis Orientação a Serviço • Construído sobre os conceitos de Serviço e Mensagens • A service is the logical manifestation of some physical or logical resources (like databases, programs, devices, humans, etc.) and/or some application logic that is exposed to the networkand • A message is a unit of communication for exchanging information. All communication between services is facilitated by the sending and receiving of messages

33. From Savas Parastatidis Serviço • Contrato • Describes the format of the messages exchanged • Defines the message exchange patterns in which a service is prepared to participate • Política • Declaratively describe service interaction requirements, quality of service, security, etc • Foco em mensagens (message-orientation)

34. From Savas Parastatidis Troca de Mensagens entre Serviços • Service-orientation (and Web Services) helps architects achieve the following properties (but do not guarantee them) • Scalability, encapsulation, maintenance, re-use, composability, loose coupling, etc.

35. From Savas Parastatidis Service-orientation vs Resource-orientation Service-orientation Resource-orientation Object-orientation

36. From Savas Parastatidis Serviço

37. From Savas Parastatidis Aplicação Orientada a Serviço

38. From Savas Parastatidis A Cluster-based Service-oriented Application

39. From Savas Parastatidis An Intranet Service-oriented Application

40. From Savas Parastatidis An Internet-scale Service-oriented Application

41. From Savas Parastatidis Serviço Web • Especificações para • Security • Orchestration • Reliability • Policies • Federation • Management • etc.

42. Peer-to-peer x Grid • P2P and Grid share a lot of common ideals • Both are services communicating by messages on shared resources • P2Ps tend to be more dynamic than Grid (Grid resources are usually quite static) • P2P applications are long-lived (i.e. everyone on the network shares a similar goal of file sharing) • Grid applications tend to be transient • P2Ps often tend to be very fault tolerant • Multiple redundancy tends to be built in • Lack of security is a significant difference between P2Ps and Grid • P2Ps don’t support the idea of VOs effectively (but nothing to stop individuals organizing themselves)

43. Grid - Níveis Global Grids • Multiple enterprises, owners, platforms, domains, file systems, locations, and security policies • Legion, Avaki, Globus Enterprise “Grids” • Single enterprise; multiple owners, platforms, domains, file systems, locations, and security policies • SUN SGE EE, Platform Multicluster Cluster & Departmental “Grids” • Single owner, platform, domain, file system and location • SUN SGE, Platform LSF, PBS Desktop Cycle Aggregation • Desktop only • United Devices, Entropia, Data Synapse Graph borrowed from A.Grimshaw

44. OGSA,WSRF,GT4

45. Serviço Web - Arquitetura

46. WS Componentes/Arq • HTTP Server • Apache HTTP Server • Application Server • Apache Tomcat • SOAP Engine • Apache AXIS • Web Service • You write this • Software stack used by GT4 WSRF Implementation

47. Serviço Web - Invocação

48. Serviço Web: Stateless

49. Serviço Web: Stateful

50. Access Grid