Grid System Issues MSI-CI 2 Meeting

1. Grid System IssuesMSI-CI2 Meeting June 29 2006 Geoffrey Fox Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401 gcf@indiana.edu http://www.infomall.org

2. Topics Covered • General Issues: Relation to P2P • Types of Grids • Why use Service Oriented Architectures • Multi-core Chips • All the world’s services • Workflow • Metadata and State • Workflow • Sensors and Filters • SOAP MPI and Communication Performance • Grids of Grids • Community Tools

3. Web services • Web Services build loosely-coupled, distributed applications, (wrapping existing codes and databases) based on the SOA (service oriented architecture) principles. • Web Services interact by exchanging messages in SOAPformat • The contracts for the message exchanges that implement those interactions are described via WSDL interfaces.

4. PortalService Security Catalog A typical Web Service • In principle, services can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) • The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python PaymentCredit Card Web Services WSDL interfaces Warehouse Shipping control WSDL interfaces Web Services

5. Philosophy of Web Service Grids • Much of Distributed Computing was built by natural extensions of computing models developed for sequential machines • This leads to the distributed object (DO) model represented by Java and CORBA • RPC (Remote Procedure Call) or RMI (Remote Method Invocation) for Java • Key people think this is not a good idea as it scales badly and ties distributed entities together too tightly • Distributed Objects Replaced by Services • Note CORBA was considered too complicated in both organization and proposed infrastructure • and Java was considered as “tightly coupled to Sun” • So there were other reasons to discard • Thus replace distributed objects by services connected by “one-way” messages and not by request-response messages

6. Some ideas to Remember • Grids are managed WebServices exchanging Messages • P2P Networks are differently managed and architected services exchanging messages • Any computer operation involves messages; not all these messages can be isolated • With services all messages are explicit and can be examined • Grid Services extend WS-* Web Service Specifications • Web Service container replaces computer • Service replaces process • A stream is an ordered set of messages • Service Internet replaces Internet: messages replace packets • (Sub)Grids replace Libraries

7. Internet Scale Distributed Services • Grids use Internet technology and are distinguished by managing or organizing sets of network connected resources • Classic Web allows independent one-to-one access to individual resources • Grids integrate together and manage multiple Internet-connected resources: People, Sensors, computers, data systems • Organization can be explicit as in • TeraGrid which federates many supercomputers; • Information Retrieval Grid which federates multiple data resources; • CrisisGrid which federates first responders, commanders, sensors, GIS, (Tsunami) simulations, science/public data • Organization can be implicit as in Internet resources such as curated databases and simulation resources that “harmonize a community”

8. Portal Services SystemServices SystemServices Application Service Middleware SystemServices SystemServices SystemServices Raw (HPC) Resources Database Typical Grid Architecture Each Blob is a Computer Program! UserServices “Core”Grid

9. Database Database Classic Grid Architecture Resources Content Access Composition Middle TierBrokers Service Providers Netsolve Security Collaboration Computing Middle Tier becomes Web Services Clients Users and Devices

10. Database Database Event/MessageBrokers Event/MessageBrokers Event/MessageBrokers Peer to Peer Grid Peers Service FacingWeb Service Interfaces Peers User FacingWeb Service Interfaces A democratic organization Peer to Peer Grid

11. Different Visions of the Grid • e-Science or Cyberinfrastructure are virtual organization Grids supporting global distributed engineering and science research (note sensors, instruments are people are all distributed) • Utility Computing or X-on-demand (X=data, computer ..) is a major computer Industry interest in Grids and this is key part of enterprise or campus Grids • Skype (Kazaa) VOIP system is a Peer-to-peer Grid (and VRVS/GlobalMMCS like Internet A/V conferencing are Collaboration Grids) • DoD’s vision of Network Centric Computing can be considered a Grid (linking sensors, warfighters, commanders, backend resources) and they are building the GIG (Global Information Grid) • Commercial 3G Cell-phones and DoD ad-hoc network initiative are forming mobile Grids • Grids support universal Globalization in life, fun, research, business

12. e-moreorlessanything and the Grid • e-Business captures an emerging view of corporations as dynamic virtual organizations linking employees, customers and stakeholders across the world. • The growing use of outsourcing is one example • e-Science is the similar vision for scientific research with international participation in large accelerators, satellites or distributed gene analyses. • The Grid integrates the best of the Web, traditional enterprise software, high performance computing and Peer-to-peer systems to provide the information technology e-infrastructure for e-moreorlessanything. • A deluge of data of unprecedented and inevitable size must be managed and understood. • People, computers, data and instruments must be linked. • On demand assignment of experts, computers, networks and storage resources must be supported

13. e-Defense and e-Crisis • Grids support Command and Control and provide Global Situational Awareness • Link commanders and frontline troops to themselves and to archival and real-time data; link to what-if simulations • Dynamic heterogeneous wired and wireless networks • Security and fault tolerance essential • System of Systems; Grid of Grids • The command and information infrastructure of each ship is a Grid; each fleet is linked together by a Grid; the President is informed by and informs the national defense Grid • Grids must be heterogeneous and federated • Crisis Management and Response enabled by a Grid linking sensors, disaster managers, and first responders with decision support

14. 1962 Licklider’s Vision “Lick had this concept – all of the stuff linked together throughout the world, that you can use a remote computer, get data from a remote computer, or use lots of computers in your job.” Larry Roberts – Principal Architect of the ARPANET

15. What is e-Science? ‘e-Science is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it.’ John Taylor Director General of Research Councils UK, Office of Science and Technology • e-Science is about developing tools and technologies that allow scientists to do ‘faster, better or different’ research

16. Some Important Styles of Grids • Computational Grids were origin of concepts and link computers across the globe – high latency stops this from being used as parallel machine • Typically Compute/File Grids where information (messages) exchanged by writing and reading files • Knowledge and Information Grids link sensors and information repositories as in Virtual Observatories or BioInformatics • Education Grids link teachers, learners, parents as a VO with learning tools, distant lectures etc. • e-Science Grids link multidisciplinary researchers across laboratories and universities • Community Grids focus on Grids involving large numbers of peers rather than focusing on linking major resources – links Grid and Peer-to-peer network concepts • Semantic Grid links Grid, and AI community with Semantic web (ontology/meta-data enriched resources) and Agent concepts • Collaboration Grids support the linkage of multiple people and electronic resources (often peer-to-peer architecture)

17. Types of Computing Grids • Running “Pleasing Parallel Jobs” as in United Devices, Entropia (Desktop Grid) “cycle stealing systems” • Can be managed (“inside” the enterprise as in Condor) or more informal (as in SETI@Home) • Computing-on-demand in Industry where jobs spawned are perhaps very large (SAP, Oracle …) • Support distributed file systems as in Legion (Avaki), Globus with (web-enhanced) UNIX programming paradigm • Particle Physics will run some 30,000 simultaneous jobs • Distributed Simulation HLA style Grids (some work) • Linking Supercomputers as in TeraGrid • Pipelinedapplications linking data/instruments, compute, visualization • Seamless Access where Grid portals allow one to choose oneof multiple resources with a common interfaces • Parallel Computing typically NOT suited for a Grid (latency)

18. Analysis and Visualization Large Disks Old Style Metacomputing Grid Large Scale Parallel Computers Spread a single large Problem over multiple supercomputers

19. Utility and Service Computing • An important business application of Grids is believed to be utility computing • Namely support a pool of computers to be assigned as needed to take-up extra demand • Pool shared between multiple applications • Natural architecture is not a cluster of computers connected to each other but rather a “Farm of Grid Services” connected to Internet and supporting services such as • Web Servers • Financial Modeling • Run SAP • Data-mining • Simulation response to crisis like forest fire or earthquake • Media Servers for Video-over-IP • Note classic Supercomputer use is to allow full access to do “anything” via ssh etc. • In service model, one pre-configures services for all programs and you access portal to run job with less security issues

20. Grid Operation Support Centre UK National Grid Service Web Services based National Grid Infrastructure

21. Towards an International Grid Infrastructure UK NGS Leeds Manchester Starlight (Chicago) US TeraGrid Netherlight (Amsterdam) Oxford RAL SDSC NCSA PSC UCL UKLight SC05 Local laptops in Seattle and UK All sites connected by production network (not all shown) Computation Steering clients Service Registry Network PoP

22. Cyberinfrastructure At Home Arecibo telescope • BOINC (Berkeley Open Infrastructure for Network Computing)(http://boinc.berkeley.edu) • Climateprediction.net: study climate change • Einstein@home: search for gravitational signals emitted by pulsars • LHC@home: improve the design of the CERN LHC particle accelerator • Predictor@home: investigate protein-related diseases • Rosetta@home: help researchers develop cures for human diseases • SETI@home: Look for radio evidence of extraterrestrial live • Etc. SETI@Home averages 138 TFLOPS on 100,000’s of computers in 100’s of countries

23. climateprediction.net Since September 2003: 95,000 registered participants in 150 countries Donated 8,000 years of computer time Completed 100,000 simulations of over 4M model years

24. Information/Knowledge Grids • Distributed (10’s to 1000’s) of data sources (instruments, file systems, curated databases …) • Data Deluge: 1 (now) to 100’s petabytes/year (2012) • Moore’s law for Sensors • Possible filters assigned dynamically (on-demand) • Run image processing algorithm on telescope image • Run Gene sequencing algorithm on compiled data • Needs decision support front end with “what-if” simulations • Metadata (provenance) critical to annotate data • Integrate across experiments as in multi-wavelength astronomy Data Deluge comes from pixels/year available

25. Data Deluged Science • In the past, we worried about data in the form of parallel I/O or MPI-IO, but we didn’t consider it as an enabler of new algorithms and new ways of computing • Data assimilation was not central to HPCC • DoE ASCI set up because didn’t want test data! • Now particle physics will get 100 petabytes from CERN • Nuclear physics (Jefferson Lab) in same situation • Use around 30,000 CPU’s simultaneously 24X7 • Weather, climate, solid earth (EarthScope) • Bioinformatics curated databases (Biocomplexity only 1000’s of data points at present) • Virtual Observatory and SkyServer in Astronomy • Environmental Sensor nets

26. Data Assimilation Information Simulation Model Datamining Ideas Reasoning ComputationalScience Data DelugedScienceComputing Paradigm Informatics

27. Field Trip Data Database ? GISGrid Discovery Services RepositoriesFederated Databases Streaming Data Sensors Database Sensor Grid Database Grid Research Education SERVOGrid Compute Grid Customization Services From Researchto Education Data FilterServices ResearchSimulations Analysis and VisualizationPortal EducationGrid Computer Farm Grid of Grids: Research Grid and Education Grid

28. SERVOGrid Requirements • Seamless Access to Data repositories and large scale computers • Integration of multiple data sources including sensors, databases, file systems with analysis system • Including filtered OGSA-DAI (Grid database access) • Rich meta-data generation and access with SERVOGrid specific Schema extending openGIS (Geography as a Web service) standards and using Semantic Grid • Portalswith component model for user interfaces and web control of all capabilities • Collaboration to support world-wide work • Basic Grid tools: workflow and notification • NOT metacomputing

29. Community Tools • e-mail and list-serves are oldest and best used • Kazaa, Instant Messengers, Skype, Napster, BitTorrent for P2P Collaboration – text, audio-video conferencing, files • del.icio.us, Connotea, Citeulike manage shared bookmarks • hotornot.com or similar sites allow you to create community resources and share them • Writely, Wikis and Blogs are powerful specialized shared document systems • ConferenceXP and WebEx share general applications • Google Scholar tells you who has cited your papers while publisher sites tell you about co-authors • Note sharing resources creates (implicit) communities • Social network tools study graphs to both define communities and extract their properties

30. Why use SOA’s • Globalization of applications: Life, Fun, Research, Business, Defense as an International collaborative activity • Globalization of Software Production: Software components including open-source made everywhere • Interoperability: in interfaces and protocol (messages) requires Web Services as only broadly supported SOA • Anti-Performance: if Moore’s law gives you a factor X, then use √X for performance, √ X for improved lifecycle (re-use) • Software Engineering: Software paradigms are ways of “packaging” modules/components/objects/methods/subroutines. Services have minimal coupling and best re-use (lowest performance). 1962 Fortran easier re-use than 2006 Java • Multicore chips: requires pervasive concurrency without side effects. Even Microsoft must be able to use 32-128 way parallelism on a chip over next 5 years

31. Intel Fall 2005 Multicore Roadmap March 2006 Sun T1000 8 core Server at <$6,000

32. Peter Kogge 1997 Performance Per Transistor • Performance data from uP vendors • Transistor count excludes on-chip caches • Performance normalized by clock rate • Conclusion: Simplest is best! (250K Transistor CPU) Normalized SPECINTS Normalized SPECFLTS Millions of Transistors (CPU) Millions of Transistors (CPU)

33. The Grid and Web Service Institutional Hierarchy 4: Application or Community of Interest (CoI)Specific Services such as “Map Services”, “Run BLAST” or “Simulate a Missile” XBMLXTCE VOTABLE CML CellML 3: Generally Useful Services and Features (OGSA and other GGF, W3C) Such as “Collaborate”, “Access a Database” or “Submit a Job” OGSA GS-*and some WS-* GGF/W3C/….XGSP (Collab) 2: System Services and Features (WS-* from OASIS/W3C/Industry) Handlers like WS-RM, Security, UDDI Registry WS-* fromOASIS/W3C/Industry 1: Container and Run Time (Hosting) Environment (Apache Axis, .NET etc.) Apache Axis.NET etc. Must set standards to get interoperability

34. Sources of Grid Technology • Grids support distributed collaboratories or virtual organizations integrating concepts from • The Web • Agents • Distributed Objects(CORBA Java/Jini COM) • Globus, Legion, Condor, NetSolve, Ninf and other High Performance Computing activities • Peer-to-peer Networks • With perhaps the Web and P2P networks being the most important for “Information Grids” and Globus for “Compute/File Grids”

35. The Essence of Grid Technology? • We will start from the Web view and assert that basic paradigm is • Meta-data rich Web Services communicating via messages • These have some basic support from some runtime such as .NET, Jini (pure Java), Apache Tomcat+Axis (Web Service toolkit), Enterprise JavaBeans, WebSphere (IBM) or GT3/4 (Globus Toolkit 3/4) • These are the distributed equivalent of operating system functions as in UNIX Shell • Called Hosting Environment or platform • W3C standard WSDL defines IDL (Interface standard) for Web Services

36. What is Happening? • Grid ideas are being developed in (at least) four communities • Web Service – W3C, OASIS, (DMTF) • Global Grid Forum (High Performance Computing, e-Science) • Enterprise Grid Alliance (Commercial “Grid Forum” with a near term focus) merged with GGF to make Open Grid Forum • Service Standards are being debated • Grid Operational Infrastructure is being deployed • Grid Architecture and core software being developed • Apache has several important projects as do academia; large and small companies • Particular System Services are being developed “centrally” – OGSA framework for this in GGF; WS-* for OASIS/W3C/Microsoft-IBM • Lots of fields are setting domain specific standards and building domain specific services • USA started but now Europe is probably in the lead and Asia will soon catch USA if momentum (roughly zero for USA) continues

37. What do Grids Add? • Grids use all of the Web Services • They address management and deployment of large distributed systems of services • Internet Scale Distributed Services • I will use Grid more simply as a composable coordinated collection of services • They address security and management issues of virtual organizations crossing multiple administrative domains • GGF is developing specific services of relevance including job management, many aspects of data and scheduling • Not much on sensors, real-time, P2P • GGF has a good process for developing new higher level specifications

38. Technical Activities of Note • Look at different styles of Grids such as Autonomic(Robust Reliable Resilient) • New Grid architectures hard due to investment required • Program the Grid – Workflow • Access the Grid – Portals, Grid Computing Environments • Critical Services Such as • Security – build message based not connection based • Notification – event services • Metadata – Use Semantic Web, provenance • Fabric and Service Management • Databases and repositories – instruments, sensors • Computing – Submit job, scheduling, distributed file systems • Visualization, Computational Steering • Network performance Low Level WS-* High Level e.g. OGSA

39. H1 H2 H3 H4 Body Container System Processing Service F1 F2 F3 F4 Container Handlers What do Web Services Prescribe? • The specify interfaces for system services (and generally useful services like database) • They specify an interface language (WSDL) for all services • They develop containers and frameworks to use to host services • They specify a message format (SOAP) for ALL messages that defines both application and system actions precisely • They imply a process be started to define domain specific services • There are multiple competing activities from Microsoft and IBM to Apache, and IU (for example) developing system and application services • Unlike for RTI and CORBA, services from different vendors should interoperate

40. Plethora of Standards • Javais very powerful partly due to its many “frameworks” that generalize libraries e.g. • Java Media Framework • Java Database Connectivity JDBC • Web Services have a correspondingly collections of specifications that represent critical features of the distributed operating systems for “Grids of Simple Services” • About 60 WS-* specifications introduced in last 2-3 years • These are low level with higher level standards such as access database (OGSA-DAI) or “Submit a job” built on top of these • Many battles both between standard bodies and between companies as each tries to set standards they consider best; thus there are multiple standards for many of key Web Service functionalities • Microsoft a key player and stands to benefit as Web Services open up enterprise software space to all participants • e.g. MQSeries (IBM) and Tibco have to change their messaging systems to support new open standards

41. The Ten areas covered by the 60 core WS-* Specifications

42. Activities in Global Grid Forum Working Groups

43. Net-Centric Core Enterprise Services

44. The Core Features/Service Areas I

45. The Core Feature/Service Areas II

46. A List of Web Services 1 • 1) Core Service Architecture • XSD XML Schema (W3C Recommendation) V1.0 February 1998, V1.1 February 2004 • WSDL 1.1 Web Services Description Language Version 1.1, (W3C note) March 2001 • WSDL 2.0 Web Services Description Language Version 2.0, (W3C under development) March 2004 • SOAP 1.1 (W3C Note) V1.1 Note May 2000 • SOAP 1.2 (W3C Recommendation) June 24 2003

47. A List of Web Services 2 • 2) Service Internet including messaging • WS-Addressing Web Services Addressing (BEA, IBM, Microsoft, SAP, Sun) in W3C consideration August 2004 • WS-MessageDelivery Web Services Message Delivery (W3C Submission by Oracle, Sun ..) April 2004 • WS-Reliability Web Services Reliable Messaging (OASIS Web Services Reliable Messaging TC) March 2004 • WS-RM Web Services Reliable Messaging (BEA, IBM, Microsoft, Tibco) v0.992 February 2005 linked to WS-Reliability in OASIS as Web Services Reliable Exchange (WS-RX) • WS-RM Policy Web Services Reliable Messaging Policy Assertion (BEA, IBM, Microsoft, Tibco) March 2006 • WS-RX Web Services Reliable Exchange (Many members) integrating previous reliability specifications • SOAP MOTM SOAP Message Transmission Optimization Mechanism (W3C) June 2004 • SOAP-over-UDP Binding of SOAP to UDP (Microsoft, BEA …) September 2004 • Many obsolete specifications like WS-Routing and Referral SOAP Routing Protocol (Microsoft) October 2001

48. Application Specific Grids Generally Useful Services and Grids Workflow WSFL/BPEL Service Management (“Context etc.”) Service Discovery (UDDI) / Information Service Internet Transport  Protocol Service Interfaces WSDL Higher Level Services ServiceContext ServiceInternet Base Hosting Environment Protocol HTTP FTP DNS … Presentation XDR … Session SSH … Transport TCP UDP … Network IP … Data Link / Physical Bit level Internet (OSI Stack) Layered Architecture for Web Services and Grids

49. WS-* implies the Service Internet • We have the classic (CISCO, Juniper ….) Internet routing the flood of ordinary packets in OSI stack architecture • Web Services build the “Service Internet” or IOI (Internet on Internet) with • Routing via WS-Addressing not IP header • Fault Tolerance (WS-RM not TCP) • Security (WS-Security/SecureConversation not IPSec/SSL) • Data Transmission by WS-Transfer not HTTP • Information Services (UDDI/WS-Context not DNS/Configuration files) • At message/web service level and not packet/IP address level • Software-based Service Internet possible as computers “fast” • Familiar from Peer-to-peer networks and built as a software overlay network defining Grid (analogy is VPN) • SOAP Header contains all information needed for the “Service Internet” (Grid Operating System) with SOAP Body containing information for Grid application service

50. A List of Web Services 3 • 3) Notification and high-level publish/subscribe information dissemination • WS-Eventing Web Services Eventing (BEA, Microsoft, TIBCO) August 2004 • WS-EventNotification (HP, IBM, Intel, Microsoft) March 2006 uses resources to manage subscriptions • WS-Notification Framework for Web Services Notification with WS-Topics, WS-BaseNotification, andWS-BrokeredNotification (OASIS) OASIS Web Services Notification TC Set up March 2004 • JMSJava Message Service V1.1 March 2002 • Different from using publish-subscribe to robustly support messaging between Web services • Bind SOAP to JMS or MQSeries