1 / 14

Real-time access to existing Grid Resources

Real-time access to existing Grid Resources. A. Stephen McGough Imperial College London On behalf of Work Package 4. Motivation. QoS is important in Workflows We need more than WS-Agreement Though WS-Agreement is very important to us

zack
Download Presentation

Real-time access to existing Grid Resources

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real-time access to existing Grid Resources A. Stephen McGough Imperial College London On behalf of Work Package 4

  2. Motivation • QoS is important in Workflows • We need more than WS-Agreement • Though WS-Agreement is very important to us • We want to allow the user to specify their QoS requirements over a workflow • Either as hard constraints • This must finish by time x • As soft constraints • This should finish by time y in 80% of cases

  3. What is GRIDCC • Adding real time control of instruments to the Grid • What makes this unique: • Instruments may only beready at specific times • Reservations • Need to ensure other Gridservices are available with instruments • Reservations / SLA’s • Real-time visualisation of live results and steering • Through Virtual Control Room (VCR) • Reservations / SLA’s Grid enabled Remote Instrumentation with Distributed Control and Computation

  4. Project Requirements • Build on top of existing Grid Middleware • In our case gLite as it was the best understood • Make use of up-coming standards • Web Services • BPEL workflows

  5. VCR ProblemSolver GRIDCC – Overview GRID Real Time QoS

  6. WfMS Architecture

  7. Planner BPEL & QoS BPEL & QoS Information Service Performance Information Agreement Service What is the Planner? • Makes a “plan” for how to execute a workflow • Can QoS requirements be met • Which resources to use • Whether reservations are required in order to meet QoS • Does not override the right of a user to request reservations • Manipulation of the workflow • The expectation for how the workflow will progress • What will be executed and where

  8. Performance Repository WS Interface WfMS Planner Observer BPEL + QoS BPEL Engine Façade F F F Agreement Service SE CE IE GRIDCC WfMS Worlflow Optimiser Validator Resource Selector Resource Pruning Reserver SEDA based architecture

  9. Plan for WfMS - Pipeline Stages Identified • XML to Object Stream • Convert the XML document into a set of objects • Object Stream to XML • Convert Objects into XML document • Reservation maker • Make reservations through the Agreement Service • Workflow fiddly bits (add in stuff we need) • Automatically deal with adding in security elements to workflow • Insert error checking into workflow, Performance logging of workflows • Workflow Validator • Workflow manipulator • Modify a workflow to improve performance • QoS validate • Validate that the workflow can meet the QoS requirements placed on it • Workflow Resource Selector • Select the resources within the Grid which will allow QoS to be met. • JDL Manipulator - Manipulate JDL documents to use resources better

  10. Must run in less than 10 mins Less than 15 mins (80% confidence) Reserve a resource for this A BPEL workflow <?xml version="1.0" encoding="UTF-8"?> <process name=“…"> <partnerLinks>… </partnerLinks> <variables> <variable element="hdns1:SimpleTriggerID" name="SimpleTriggerID"/>… </variables> <flow> <sequence name="WSIIETestSequence"> <receive createInstance="yes" name="WSIIETestInitiate" operation="IETest" partnerLink="initiate_IETest_PL“…/> <sequence name="WSIIETestInitialiseAndGetStatus"> <invoke inputVariable="getDTInstanceRequestVar" name="WSIIE-getDTInstance" operation="getDTInstance“…> <target linkName="L5"/> <source linkName="L6"/> </invoke> <invoke inputVariable="getDTStatusRequestVar“…>… </invoke> <assign name="GetStatusOutput"> <target linkName="L8"/> <copy> <from part="parameters" query="" variable=""/> <to variable="StatusValue"/> </copy> </assign> </sequence> <switch> <case condition="bpws:getVariableData('StatusValue')!= bpws:getVariableData('StatusValueRunning')"> <sequence name="InitConfigureAndStartIESequence"> <while condition="bpws:getVariableData('StatusValue')!= bpws:getVariableData('StatusValueRunning')" name="UntilStatusIsRunning"> <sequence> … </sequence> </while> </sequence> <reply …/> </sequence> </flow> </process> Using XPath pointers into BPEL document

  11. The whole (Work)flow All 2Ghz Boxes Total Time 100s 100% Reliable resources Simple Example <?xml version="1.0" encoding="UTF-8"?> <QoSRequirements xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\asm\Desktop\QoSRequirements.xsd"> <QoSConstraint> <XpathReference>/process/flow</XpathReference> <CPUSpeed>2048000</CPUSpeed> <MaxDurationTime>100</MaxDurationTime> <Reliability>100</Reliability> </QoSConstraint> </QoSRequirements>

  12. Refer to multiple parts of document <?xml version="1.0" encoding="UTF-8"?> <QoSRequirements xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\asm\Desktop\QoSRequirements.xsd"> <QoSConstraint> <XpathReference>/process/flow/sequence[1]/sequence[1]/invoke[1]</XpathReference> <XpathReference>/process/flow/sequence[1]/sequence[1]/invoke[2]</XpathReference> <CPUSpeed>2048000</CPUSpeed> <MaxDurationTime>100</MaxDurationTime> <Reliability>100</Reliability> </QoSConstraint> </QoSRequirements>

  13. <?xml version="1.0" encoding="UTF-8"?> <QoSRequirements xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\asm\Desktop\QoSRequirements.xsd"> <QoSConstraint> <XpathReference>/process/flow/sequence[1]/sequence[1]/invoke[1]</XpathReference> <ReservationRequired> <StartTime>2006-10-02T23:00:00.0Z</StartTime> <EndTime>2006-10-02T23:59:59.0Z</EndTime> <EndpointName>cpu1.doc.ic.ac.uk</EndpointName> </ReservationRequired> <CPUSpeed>1024000</CPUSpeed> <MaxDurationTime>80</MaxDurationTime> <Reliability>100</Reliability> </QoSConstraint> <QoSConstraint> <XpathReference>/process/flow/sequence[1]/sequence[1]/invoke[2]</XpathReference> <CPUSpeed>2048000</CPUSpeed> <MaxDurationTime>20</MaxDurationTime> <Reliability>50</Reliability> </QoSConstraint> </QoSRequirements> <ReservationMade> <StartTime>2006-10-02T23:00:00.0Z</StartTime> <EndTime>2006-10-02T23:59:59.0Z</EndTime> <EndpointName>cpu3.doc.ic.ac.uk</EndpointName> <Token>dveinvunvuin42235454</Token> </ReservationMade> Make a reservation through Agreement Service (WS-Agreement) Store result here

  14. Questions

More Related