1 / 39

Part 3- Mapping to the Grid

A simple example using Semantic Web tools and technologies to describe grid resources ... Effective Grid resource description and matching demands Semantic Web solutions ...

Roberta
Download Presentation

Part 3- Mapping to the Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    Slide 1:Mapping to the Grid David De Roure

    Assume that you know what we might mean by Semantics (dangerous!) Before we talk about the semantic Grid, we must talk about the Grid and why it needs some semantics.Assume that you know what we might mean by Semantics (dangerous!) Before we talk about the semantic Grid, we must talk about the Grid and why it needs some semantics.

    Slide 2:Overview

    Tutorial example A simple example using Semantic Web tools and technologies to describe grid resources Storing and querying with triplestores Semantic Web applied in the Grid underware Challenges to a comprehensive solution Moving forward in GGF

    Slide 3:Ontology-based Resource Matching: The Grid Meets the Semantic Web

    Hongsuda Tangmunarunkit Carl Kesselman Center for Grid Technologies Stefan Decker Intelligence Division Information Sciences Institute University of Southern California

    Slide 4:Condor Matchmaker

    Exact syntactic matching Examples: Condor Matchmaker, PBS Use symmetric syntax (i.e., attribute-value pairs) to describe resources and requests Type=“Machine”; Type=”Job”; Constraints are specified by conjunctions and disjunctions of arithmetic/string comparisons and set membership Boolean operations Memory > 1000

    Slide 5:Example

    Type=“Job”; Owner=“u2”; Constraint= other.Type==“Machine” && Memory>500 && OpSys==“Solaris251”; Rank=other.Memory Type=“Machine”; Name=“m2”; Memory=1000; OpSys=“Solaris251”; Grp1={“u1”,”u2”}; Constraint=member(other.Owner, Grp1) Type=“Machine”; Name=“m1”; Memory=1500; OpSys=“Redhat7.3”; Grp1={“u1”,”u2”}; Constraint=member(other.Owner, Grp1) Machine=“m2” Matchmaker Resource Requesters (users/programs)

    Slide 6:Finding the Unix machine

    A machine’s operating system is described as “SunOS” or “Linux.” To query for a machine that is “Unix” compatible, a user either has to: Explicitly incorporate the Unix compatibility concept into the request requirements by requesting a disjunction of all Unix-variant operating systems e.g., (OpSys=“SunOS” OpSys=“Linux”) The disjunctive requirements become unwieldy as more abstract concepts are developed. Wait for all interesting resources to advertise their operating system as Unix as well as either Linux or SunOS e.g., (OpSys= “SunOS,” “Unix”), and then express a match as set-membership of the desired Unix value in the OpSys value set e.g., hasMember(OpSys, “Unix”). The advertisements become more complex and all resources must be updated before a match can occur.

    Slide 7: Solution 1

    Type=“Job”; Owner=“u2”; Constraint= other.Type==“Machine” && Memory>500; (OpSys==“Solaris251” || OpSys==“Redhat7.3”) Rank=other.Memory Type=“Machine”; Name=“m2”; Memory=1000; OpSys=“Solaris251”; Grp1={“u1”,”u2”}; Constraint=member(other.Owner, Grp1) Type=“Machine”; Name=“m1”; Memory=1500; OpSys=“Redhat7.3”; Grp1={“u1”,”u2”}; Constraint=member(other.Owner, Grp1) Machine=“m2” Matchmaker Resource Requesters (users/programs)

    Slide 8: Solution 2

    Type=“Job”; Owner=“u2”; Constraint= other.Type==“Machine” && Memory>500; member(“Unix”, other.OpSys) Rank=other.Memory Type=“Machine”; Name=“m2”; Memory=1000; OpSys={“Solaris251”, “Unix”}; Grp1={“u1”,”u2”}; Constraint=member(other.Owner, Grp1) Type=“Machine”; Name=“m1”; Memory=1500; OpSys={“Redhat7.3”, “Unix”}; Grp1={“u1”,”u2”}; Constraint=member(other.Owner, Grp1) Machine=“m2” Matchmaker Resource Requesters (users/programs)

    Slide 9:Ontologies

    Developing 3 Ontologies using RDF-Schema Resource Ontology Describe resource components, their properties and relationships Classes: ComputerSystem, OperatingSystem Relationships: RunningOS Based on Common Information Model (CIM) Resource Request Ontology Describe applications, characteristics and resource requirements, e.g., JobType=“MPI” Policy Ontology Resources specify their usage and authorization policies

    Slide 10:Resource Ontology

    Currently focus on Computation Resources

    Slide 11:Example: Resources

    Slide 12:Example: Request

    // query FORALL X,Y <- X[matches->Y]@match(advertisement,gridBackground,gridOntology). X = Request1, Y = Almaak.usc.edu

    Slide 13:From Pizza to the Grid

    Let’s apply Semantic Web thinking to a fundamental Grid problem that everyone recognises – the description of the capabilities of Grid resources. In a full length Semantic Web tutorial we usually build a pizza ontology. The Semantic Grid version would be a grid resource ontology instead…

    Slide 14:Pizza Ontology

    Platform Operating system Unix Windows Cluster Datastore

    Slide 15:Tools

    This example uses OilEd the "NotePad" of ontology editors, offering enough functionality to allow users to build ontologies see http://oiled.man.ac.uk The FACT reasoner is used to classify the ontology Can also illustrate this in Racer, with instances Semantic Web inference engine “Reasoner for ABoxes and Concept Expressions Renamed” See http://www.sts.tu-harburg.de/~r.f.moeller/racer/

    Slide 17:Pizza Problems

    If we actually did this exercise we might encounter some common problems: Failure to make all information explicit Use of universal existential restrictions as the default Open world reasoning Effect of range and domain constraints as axioms Trivial satisfiability of universal restrictions Difference between defined and primitive classes Errors in understanding common logical constructs Expecting classes to be disjoint by default Difficulty with subclass axioms used for implication

    Slide 18:Using triplestores

    When we have decided how to describe grid resources… Where do we keep this information? How do we query it? How do we update it?

    Slide 19:Choosing triplestores

    Jena http://www.hpl.hp.com/semweb/ 3store triplestore.aktors.org Scaleable 10 -> 15 -> 25M triples RDQL or OKBC Linearish performance Knows source (provenance)

    http://triplestore.aktors.org/SemanticWebChallenge/

    Slide 21:Towards Grid Google ?

    A triplestore describing Grid resources would provide a kind of “Grid Google” Still need to differentiate between the results (by machine) This could be a job for WS-Agreement, Condor, … Can this matchmaking be achieved with Semantic Web technologies? Where do we draw the line?

    Slide 22:Common Information Model

    In general a CIM defines information available for sharing among multiple business processes and the applications that support them The n-squared problem: If N is the number of applications, I is the number of interfaces, and each application has one interface in each direction with every other application, then I = 2 * N * (N-1) = 2N2 - 2

    Slide 23:Common Information Model

    There have been various CIM efforts The Data Management Task Force (DMTF) specifies an object-oriented framework for implementing the CIM Defined as CIM provides a common definition of management information for systems, networks, applications and services, and allows for vendor extensions. CIM’s common definitions enable vendors to exchange semantically rich management information between systems throughout the network.

    Slide 24:Requirements 1 - GRIP

    EU Funded FP5 Project as part of Information Society Technologies Programme IST 2001-32257 http://www.grid-interoperability.org/

    EuroGrid: Meteo-Grid

    Slide 26:Requestor & Provider Spaces

    Resource requestor space (RR), in terms of what the user wants: e.g. Relocatable Weather Model, 10^6 points, 24 hours, full topography. Resource Provider space (RP), 128 processors, Origin 3000 architecture, 40 Gigabytes Memory, 1000 Gigabytes disk space, 100 Mb/s connection. We may even forward on requests from one resource provider to another, recasting of O3000 job in terms of IA64 cluster, gives different resource set. Linkage and staging of different stages of workflow require environmental support, a hosting environment. We can have multiple offers in RP space for the same RR values

    Slide 27:Grid Resource Description

    Two Independent Grid Systems Unicore (http://www.unicore.org/) Globus (http://www.globus.org/) Both Need to Describe Systems that run Compute Jobs Very Different Description Languages Unicore’s Resource model, part of the AJO Framework Globus’s GLUE Schema (DataTAG, iVGDL) for GT2 and GT3 For interoperability, we want to take a Unicore job and run it on Globus resources Therefore, we need to translate the Job’s Resource Requirements between the two Systems

    Slide 28:Insight from GRIP

    The process of building the ontologies gave insight into the fundamental differences in conceptualisation and hence the barriers to interoperability Globus (via the GLUE schema) attempts to model the abstract structure of the resources “This is what we are, this is what we can do” Unicore AJO abstracts the request for resources “I want this work done in time for this event.”

    http://www.unigrids.org/

    Slide 30:Requirements 2 - GRIA

    Objectives: make the Grid usable for business and industry! Participants: University of Southampton IT Innovation Intelligence, Agents and Multimedia NTUA Dolphin CESI Kino www.gria.org

    Slide 31:GRIA Applications

    Structural Identification ENEL case study Structural analysis, e.g. finite elements model updating Outsourcing/Insourcing scenarios Virtual Digital Studio KINO case study Post-production tasks scene rendering image enhancement Outsourcing/Collaborative scenarios

    Slide 32:Example 1 Querying 3rd parties

    I want to run a very large finite-element analysis, 100000 DOFs, fully dynamic treatment, and I need the result in one hour. My GRIA client can figure out 10 workload parameters and establish which GRIA service providers can deliver the necessary computation. But none are able to finish the work in the time I need! I know EPCC carry FEM codes running under GT3, and Stuttgart supplies services over UNICORE to several automotive customers. My needs are small by their standards – but do they have enough spare capacity to meet my deadline, and can I use them?

    Slide 33:Querying third party providers

    GRIA QoS is based on: client-side workload estimation server-side estimation of capacity to deliver the work Need application-independent workload representation related to resource capacity Use 10 load parameters: CPU, memory, disk, data transfer, network transfer predicted from application parameters

    Slide 34:Example 2 Multiple related workloads

    I work for KINO, and I am making a commercial. I need a 60-second sequence showing athletes preparing for the final day of competition, and my client wants several complete scenarios featuring different athletic events from which to edit the final cut. I need to render 4500 high-definition frames based on an architect’s model. I need the results first thing tomorrow morning, and my budget is only $15 for this job. Three service providers can meet my needs. One will charge $0.10 per frame, but can handle 25000 frames for overnight delivery. There are cheaper suppliers, and my in-house systems can do it for free (but only if nothing else is running).

    Slide 35:Example 3 Embedding in processes

    Our friend from KINO is desperate. The usual supplier can’t take on the excess work after all, and he needs to find other service providers. There are a couple of new guys on the block, but they don’t use the normal tender/order/deliver/invoice process: The National Technical University of Athens provides access to rendering codes. They post Condor class ads describing resources and assign them to the highest bidder using an auction protocol. The local cable company provides rendering services on their digital video on demand servers during off-peak periods. However, they allow access only to signed-up clients, who can then submit jobs with no QoS negotiation. Delivery is on a “best efforts” basis, and tariffs depend on when each computation is submitted and when the result is needed.

    Slide 36:Requirements

    Resource descriptions Job descriptions Negotiation of Service Level Agreements Policy Accounting …

    Slide 37:Moving forward in GGF

    We understand the requirements We need to build the ontologies We would like a GGF Working Group to do this!

    Slide 38:GGF

    Human endeavor is caught in an eternal tension between the effectiveness of small groups acting independently and the need to mesh with the wider community. A small group can innovate rapidly and efficiently, but this produces a subculture whose concepts are not understood by others. Coordinating actions across a large group, however, is painfully slow and takes an enormous amount of communication. The world works across the spectrum between these extremes, with a tendency to start small - from the personal idea - and move toward a wider understanding over time. An essential process is the joining together of subcultures when a wider common language is needed. Tim Berners-Lee, James Hendler and Ora Lassila. Scientific American (May 2001)

    Slide 39:Summary

    Effective Grid resource description and matching demands Semantic Web solutions Need further engagement with other activities in GGF in order to express descriptions using Semantic Web technologies

More Related