Grid Computing. School of Engineering Young Joo, Han. Contents. Grid Computing Comparison with other tech Grid Architecture Fabric layer Connectivity layer Resource layer Collective layer Application layer Grid Category Computational Grid Data Grid Access Grid. scale. Multi-tier
Grid Computing School of Engineering Young Joo, Han
Contents • Grid Computing • Comparison with other tech • Grid Architecture • Fabric layer • Connectivity layer • Resource layer • Collective layer • Application layer • Grid Category • Computational Grid • Data Grid • Access Grid
scale Multi-tier Server-side Grid Computing Component programming Client-server Classes Application complexity Object-oriented programming monolithic Structured programming 1970 1980 1990 2000 Time (years) Software Trends
Virtual Servers, Storage and Instruments Grid Middleware Distributed Physical Servers and Storage Grid Computing • Grid computing offers a model for solving massive computational problems by making use of the unused CPU cycles of large numbers of disparate, often desktop, computers treated as a virtual clusterembedded in a distributed telecommunications infrastructure. • The coordinated, transparent and secure sharing of IT resources across geographically distributed sites • Grids are built on standards-based interoperability to deliver integrated solutions that allow virtual collaboration
Grid Enable • Sharing of Resources • – Virtual Organizations and Collaboration • Management of Resources • - Quality of Service and Optimization • Access to Resources • - On Demand Computing and Utility Models Source: IBM
Grid Technology • Natural evolution of distributed systems and the Internet. • Middleware supporting network of systems to facilitate sharing, standardization and openness. • Infrastructure and application model dealing with sharing of compute cycles, data, storage and other resources. • Move towards delivering “computing” to masses similar to other utilities (electricity and voice communication). • Currently used for high performance computing however the trend is towards Service Oriented Applications (SOA).
Open Grid Services Architecture(Integration of Grid and Web services) share access manage Continuous Availability Applications on demand Resources On demand Secure and Universal access Global accessibility Business integration Vast resource scalability Web Services Grid Protocols
Application Application Internet Protocol Architecture “Coordinating multiple resources”: ubiquitous infrastructure services, app-specific distributed services Collective “Sharing single resources”: negotiating access, controlling use Resource “Talking to things”: communication (Internet protocols) & security Connectivity Transport Internet “Controlling things locally”: Access to, & control of, resources Fabric Link Layered Grid Architecture(By Analogy to Internet Architecture) Slide courtesy of C. Kessleman Cal(IT)2 Presentation
Layered Grid Architecture Application Application Collective Internet Protocol Architecture Resource Grid Protocol Architecture Transport Connectivity Internet Fabric Link
Fabric Layer • Fabric layer: Provides the resources to which shared access is mediated by Grid protocols. • Example: computational resources, storage systems, catalogs, network resources, and sensors. • Fabric components implement local, resource specific operations. • Richer fabric functionality enables more sophisticated sharing operations. • Sample resources: computational resources, storage resources, network resources, code repositories, catalogs.
Connectivity Layer Application Collective Resource Grid Security Infrastructure GSI Grid Protocol Architecture Connectivity Nexsus Interface Fabric
Connectivity Layer • Communicating easily and securely. • Connectivity layer defines the core communication and authentication protocols required for grid-specific network functions. • This enables the exchange of data between fabric layer resources. • Support for this layer is drawn from TCP/IP’s IP, TCL and DNS layers. • Authentication solutions: single sign on, etc.
Resource Layer Grid Resource Access Management (GRAM) Application Resource Management Collective Grid Resource Information Protocol (GRIP) Resource Grid Protocol Architecture Grid Resource Registration Protocol (GRRP) Connectivity Grid Information Services GridFTP Fabric Data Transfer
Resources Layer • Resource layer defines protocols, APIs, and SDKs for secure negotiations, initiation, monitoring control, accounting and payment of sharing operations on individual resources. • Two protocols information protocol and management protocol define this layer. • Information protocols are used to obtain the information about the structure and state of the resource, ex: configuration, current load and usage policy. • Management protocols are used to negotiate access to the shared resource, specifying for example QoS, advanced reservation, etc.
Collective Layer Application Directory Services Collective Data Replication Services Resource Grid Protocol Architecture Monitoring Services Connectivity Scheduling and Brokering Services Fabric
Collective Layer • Coordinating multiple resources. • Contains protocols and services that capture interactions among a collection of resources. • It supports a variety of sharing behaviors without placing new requirements on the resources being shared. • Sample services: directory services, coallocation, brokering and scheduling services, data replication service, workload management services, collaboratory services.
Applications Layer • These are user applications that operate within VO environment. • Applications are constructed by calling upon services defined at any layer. • Each of the layers are well defined using protocols, provide access to useful services. • Well defined APIs also exist to work with these services. • A toolkit Globus implements all these layers and supports grid application development.
Presenter mic Distributed Super Computing Computational Grid High-Throughput Computing Grid System Presenter camera Data Grid On Demand Access Grid Cooperative work Environment Ambient mic (tabletop) Multimedia Audience camera Categorization
Computational Grid • Computational grid is a hardware and software infrastructure that provide dependable, consistent, pervasive and inexpensive to access to high-end computational capabilities • Infrastructure because computational grid is large-scale pooling of resources, whether compute cycle, data, sensors, or people • Computational grid is analogous to electric power grid
Example (Butterfly.net) • Massively Multiplayer Online Game • Very complex, wide virtual world, many buildings, rooms, polygons, textures, characters, and so on… • Millions of users invests enormous amounts of time, energy, money. • $10 billion in 2001 to $18 billion in 2005 (US only) • Lineage (NCSoft, South Korea). The biggest MMG game in the world, the most number of users. (1.2 millions users pay for it each month.) • EverQuest(US), a half-million subscrivers.
Technical problems • Inherently issuing problems • MMGs are often unavailable for hours at a time • MMGs suffer from lag • MMGs offer a minimal set of interactions. • Reasons • Difficulties on development • network-based game VS. standalone PC game • “Client-server protocol • Difficulties on load estimation • Server maintenance and reconfiguration
Solutions • Low cost and high performance equipment • Intel processor + Linux OS • Distributed game server architecture on Grid environment and multicast-mesh over User Datagram Protocol(UDP) • Security, Reliability, Scalability, Load balancing, high speed and high quality game, • No down time, No lag • Cost effective, • No risk
Data Grid • A data grid is a grid computing system that deals with data - the controlled sharing and management are large amounts of distributed data. These are often, but not always, combined with computationalgrid computing systems. • These applications require widely distributed access to data by many people in many places. • The data grid creates virtual collaborative environments that support distributed but coordinated scientific and engineering research • Example: Medical X-ray DB(telepacs system)
Access Grid • Access Grid is a collection of resources and technologies that enable large format audio and video based collaborations: • The Access Grid is an ensemble of resources including multimedia large-format displays, presentation and interactive environments, and interfaces to Grid middleware and to Visualization environments
Example (Share of 3D Objects & Collaborative Interaction) Remote Screen KIST Screen
Grid Past, Present, Future • Past • Origins and broad adoption in eScience, fueled by open source Globus Toolkit • Present • Rapidly growing commercial adoption • Open Grid Services Architecture (OGSA) • Future • Key enabler of new applications & industries based on resource virtualization and distributed service integration
Reference • www.ggf.org • www.wordiq.com • www.emergentgametech.com • www.accessgrid.org • www.gridcenter.or.kr • Analysis of state Management in Web Services and Various Extensions (Ai Ting, Wang Caixia, Xie Young) • Grid Computing (Wieley) • Etc.