1 / 17

Workflow Interoperability Challenge

SHIWA: Is the Workflow Interoperability a Myth or Reality PUCOWO, 07-08. June 2011, London Gabor Terstyanszky, Tamas Kiss, Tamas Kukla University of Westminster. Workflow Interoperability Challenge. SHIWA Aims and Objectives. E-scientists

tale
Download Presentation

Workflow Interoperability Challenge

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SHIWA: Is the Workflow Interoperability a Myth or Reality PUCOWO, 07-08. June 2011, LondonGabor Terstyanszky, Tamas Kiss, Tamas Kukla University of Westminster

  2. Workflow Interoperability Challenge

  3. SHIWA Aims and Objectives • E-scientists • Aims:seamlessly “recycle” existing workflows of different workflow systems • Objectives: • To be able to search and browse available workflows • To be able to execute non-native workflows and create and submit embedded workflows • Workflow and workflow engine developers • Aims:publish workflows to enable their sharing • Objectives: • To be able develop and test workflows • To be able upload and share workflows

  4. SHIWA Workflow Interoperability Solution

  5. SHIWA concept of coarse-grained workflow interoperability 0 1 NativeJob • WFs are black boxes and we do not interpret them • Only inputs and outputs are known • WFs are executed by their engines remotely • Such WFs can be embedded as jobs into WFs of other systems forming embedded or heterogeneous meta WFs. 2 3 0 1 2 0 1 2 MOTEURWF AskalonWF 3 3 0 1 NativeJob 2 5

  6. SHIWA Simulation Platform (SSP) • SHIWA Simulation Platform v0 (SSP v0): October 2010 • portal: PGRADE 2.4.1 • repository: GEMLCA Legacy Repo • workflow: native – PGRADE • workflow engines are legacy codes and workflows are their parameter • SHIWA Simulation Platform v1 (SSP v1): March 2011 • repository: SHIWA Workflow Repo • both workflow engines and workflows are legacy codes • SHIWA Simulation Platform v2 (SSP v2): March 2012 • portal: WS-PGRADE 3.3 • repository: seamless data transfer among the GEMLCA and SHIWA repos • workflow: native – ASKALON + MOTEUR

  7. SSP v1: Architecture SHIWA Science Gateway SHIWA VO SHIWA Portal SHIWA Repository local cluster Globus DCI gLite DCI WF1 WFn GEMLCA admin Triana WE Taverna WE WF1 WFm WE1 WEp ASKALON WE MOTEUR WE GEMLCA Repository PGRADE Workflow editor Kepler WE MOTEUR WE GWES WE PGRADEWorkflow engine ASKALON WE pre-deployed-WEs GEMLCA with GIB GEMLCA Service

  8. SSP v1: SHIWA Science Gateway • SHIWA Portal: • P-GRADE 2.4 portal technology • certificate/proxy and DCI resource management • access to different DCI information systems • integrated with the P-GRADE Workflow System (used as native workflow engine) • administration of GEMLCA services • GEMLCA Service: • converts legacy applications such as workflows and workflow engines into Grid services • invokes locally or remotely pre-deployed workflow engines or submits workflow engines to local or remote resources to execute workflows • GEMLCA Repository • workflow engine (WE) and workflow (WF) data supporting execution • SHIWA Repository: • create, add, edit and delete workflow metadata • upload and download workflows with their implementations and configurations. SHIWA Science Gateway SHIWA Portal SHIWA Repository WF1 WFn GEMLCA admin WF1 WFm WE1 WEp GEMLCA Repository PGRADE Workflow editor PGRADEWorkflow engine GEMLCA with GIB GEMLCA Service

  9. SHIWA Repository Data Structure • Workflow description • plain text describing inputs and outputs and explains what the workflow does • Workflow implementations • all implementation files or references to these files • data/metadata necessary to run the workflow on its associated workflow engine. • Workflow configurations • pre-defined input files and parameters or references to these, and other data/metadata of a workflow. • Workflow engines • files and other data/metadata necessary to execute a workflow engine on a grid site or references to them

  10. SHIWA Workflow Repository http://dev17-portal.cpc.wmin.ac.uk:8080/shiwa-repo/

  11. SSP v1: User scenarios - Workflow engine upload SHIWA Science Gateway SHIWA Portal SHIWA Repository Actor: WE developer Frequency: once for a particular version of a WE Process: step 1- create and upload workflow engine’s legacy code description (LCID) using the GEMLCA admin portlet. step 2 - upload the workflow engine’s binary to the GEMLCA Repository (WE1, …WEp) WF1 WFn s1 create LCID GEMLCA admin WE+ LCID upload WE s2 WF1 WFm WE developer WE1 WEp GEMLCA Repository PGRADE Workflow editor PGRADEWorkflow engine GEMLCA with GIB GEMLCA Service

  12. SSP v1: User scenarios - Workflow upload SHIWA Science Gateway Actor: WF developer Frequency: once for a particular version of a WF Process: step 1 - create and upload the workflow’s LCID description using the GEMLCA admin portlet step 2 - upload the workflow descriptor file to the GEMLCA Repository (WF1, …WFm). step 3 - define the workflow metadata using the SHIWA Repository GUI step 4 - upload the workflow descriptor file to SHIWA Repository (WF1, …WFn). SHIWA Portal SHIWA Repository create WF metadata s3 WF1 WFn upload WF s4 s1 create LCID GEMLCA admin WF + LCID s2 upload WF WF1 WFm WF developer WE1 WEp GEMLCA Repository PGRADE Workflow editor PGRADEWorkflow engine GEMLCA with GIB GEMLCA Service

  13. SHIWA VO SHIWA Science Gateway local cluster Globus DCI gLite DCI SHIWA Portal SHIWA Repository WF1 WFn Triana WE search WF GEMLCA admin Taverna WE s1 MOTEUR WE WF1 WFm s2 WE1 WEp WF list ASKALON WE Kepler WE edit WF GEMLCA Repository s2 PGRADE Workflow editor ASKALON WE MOTEUR WE GWES WE s4 WE + WF e-scientist s3 PGRADEWorkflow engine GEMLCA with GIB pre-deployed-WEs s5 invoke WE GEMLCA Service SSP v1: User scenarios – workflow editing & execution submit WE s5 step 1 – browse/search the SHIWA repository and select workflows to execute step 2 - create embedded workflows by adding the selected non-native workflows using their IDs step 3 - P-GRADE workflow engine submits the embedded workflow through GEMLCA Service step 4 - GEMLCA Service combines the workflow and its corresponding workflow engine. step 5 - the GEMLCA Service invokes the pre-deployed workflow engine to execute the workflow or submits the workflow engine with the workflow

  14. SSP v1 – How to get access? • Register with the SHIWA VO: • aim: to get authorisation to access GEMLCA resource, WF engines, compute and data resources • https://cclcgvomsli01.in2p3.fr:8443/voms/shiwa-workflow.eu • Get an account to the SHIWA portal • aim: to get access to the portal • Register with the SHIWA repository • aim: to be authorised to upload and modify workflows • repository can be browsed and workflows can be downloaded without account • Detailed how-to-get-access guide: • http://www.cpc.wmin.ac.uk/cpcsite/index.php/Other_projects:SHIWA_SSPv1

  15. SSP v2: Architecture c Unicore DCI ARC DCI SHIWA Science Gateway gLite DCI Globus DCI SHIWA Portal SHIWA Repository WF1 WFn GWES WE Kepler WE GEMLCA Repository GEMLCA admin WF1 WFm PGRADE WE Triana WE WE1 WFp Taverna WE PGRADE Workflow editor GEMLCA submitter DCI Bridge MOTEUR WE ASKALON WE PGRADEWorkflow engine ASKALON GEMLCA UI MOTEUR GEMLCA UI GEMLCA client pre-deployed-WEs

  16. SSP v2: Science Gateway • SHIWA Portal: • WS-PGRADE 3.3 • DCI Bridge: • access to multiple DCIs • SHIWA Repository: • seamless data transfer among the GEMLCA and SHIWA repositories SHIWA Science Gateway SHIWA Portal SHIWA Repository WF1 WFn GEMLCA Repository GEMLCA admin WF1 WFm WE1 WFp PGRADE Workflow editor GEMLCA submitter DCI Bridge PGRADEWorkflow engine GEMLCA client 16

  17. SSP v2: User scenarios – workflow editing & execution DCIs c Unicore DCI ARC DCI SHIWA Science Gateway gLite DCI Globus DCI SHIWA Portal SHIWA Repository search WF WF1 WFn s1 GWES WE Kepler WE GEMLCA Repository GEMLCA admin WF1 WFm PGRADE WE Triana WE edit WF WE1 WFp Taverna WE s2 WE + WF s4 e-scientist PGRADE Workflow editor GEMLCA submitter MOTEUR WE ASKALON WE s5 invoke WE s3 PGRADEWorkflow engine ASKALON GEMLCA UI MOTEUR GEMLCA UI GEMLCA client access non-native WF s3 pre-deployed-WEs

More Related