1 / 20

New developments of gUSE & WS-PGRADE to support e-science gateways

New developments of gUSE & WS-PGRADE to support e-science gateways. Peter Kacsuk and Miklos Kozlovszky MTA SZTAKI - Computer and Automation Research Institute of the Hungarian Academy of Sciences EGI User Forum 2011 Vilnius, Lithuania 11.04.2011. www.lpds.sztaki.hu. Content.

swarthout
Download Presentation

New developments of gUSE & WS-PGRADE to support e-science gateways

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. New developments of gUSE & WS-PGRADE to support e-science gateways Peter Kacsuk and Miklos Kozlovszky MTA SZTAKI - Computer and Automation Research Institute of the Hungarian Academy of Sciences EGI User Forum 2011 Vilnius, Lithuania 11.04.2011. www.lpds.sztaki.hu

  2. Content • WS-PGRADE & gUSE in a nutshell • Basic and new services • Case studies • How to access

  3. HP-SEE P-GRADE Portals around the globe • National Grid portals (UK, Ireland, Croatia, Turkey, Spain, Belgium, Malaysia , Kazakhstan, Switzerland, Australia, ClGrid, etc.) • HP-SEE/SEE-GRID-SCI, BalticGrid • US Open Science Grid, TeraGrid • IDGF (International Desktop Grid Federation) • SHIWA Simulation Platform • Economy-Grid, Swiss BioGrid, Bio ésBiomed Virtual Organisations, MathGrid, etc.

  4. 2011 2010 2009 2008 Open source from 2008 stats: P-GRADE Portal software family P-GRADE Portal 2.4 GEMLCA Grid Legacy Code Arch. P-GRADE Portal 2.5 NGS P-GRADE portal GEMLCA, storage concepts P-GRADE Portal 2.8 Basicconcepts WS-PGRADEPortal 3.1 P-GRADE Portal 2.9.1 WS-PGRADEPortal 3.2 P-GRADE Portal 2.10 WS-PGRADEPortal 3.3

  5. Motivations of creating gUSE • To overcome (most of)the limitations of P-GRADE portal: • To provide better modularity  to replace any service • To improve scalability  to millions of jobs • To enable advanced dataflow patterns • To interface with wider range of resources • To separate Application Developer view from Application User view WS-PGRADE (Web Services Parallel Grid Runtime and Developer Environment) and gUSE (Grid User Support Environment) architecture

  6. gUSE and WS-PGRADE architecture Graphical User Interface: WS-PGRADE Gridsphere  LIFERAY Large set of portlets gUSE Filestorage Workflowstorage Filestorage gUSEinformationsystem Autonomous Services: high level middleware service layer WorkflowEngine Applicationrepository Submitters Submitters Submitters Submitters Meta-broker Logging Resources: middleware service layer Local resources, service grid VOs, Desktop Grid resources, Web services, Databases

  7. WS-PGRADE/gUSE • Creating complex workflow and parameter sweeps • Seamless access to various types of resources • clusters, • service grids, • desktop grids, • databases. • Scalable architecture • Advanced data-flows and workflow concept • Creating complex applications using embedded workflows, legacy codes • Comfort features • Separated views for application developers and end-users • Community components from workflow repository www.wspgrade.hu

  8.              WS-PGRADE in a nutshell • General purpose, workflow-oriented portal. Supports the development and execution of workflow-based applications • Based on Liferay • Services supported by the portal: • New functionalities • Web services • DB connectors • Embedded workflows • Job level PS • Conditional jobs • Recursive graph • Multi-generator • Multi-collector • CROSS product PS • DOT product PS             Simultaneous use of various clusters, service and desktop Grids at workflow level

  9. Latest developments of gUSE • gUSE service orchestration • direct remote access • ASM (Application Specific Module) • JAVA API • New submitters • ARC • UNICORE + SAML (Security Assertion Markup Language) support • LFS support

  10. gUSE service orchestration • Accessing gUSE services remotely • direct remote access • ASM (Application Specific Module) • JAVA API Prog. Env. (JAVA API) Remote access module (JAVA API) Chemaxon Ltd. WS-PGRADE MOSGRID/x Remote client (CLI) ASM SHIWA gUSE Remote access API

  11. WS-PGRADE in European and national projects • CancerGrid (EU FP6) • ProSim (UK) • EDGeS and EDGI (EU FP7) • MosGrid (D-Grid) • Swiss Proteomics Portal • HP-SEE (EU FP7) • SHIWA (EU FP7) • SCI-BUS (EU FP7)

  12. The CancerGrid portal for anti-cancer drug research Workflow execution Workflow development & configuration Integrated components of CancerGrid portal Algorithms configuration Molecule database browser Structure viewer

  13. ProSim workflow in g-USE • UK project for Protein Molecule Simulation on the Grid • a combination of GEMLCA and standard g-USE jobs • Executed on 5 different sites of the UK NGS • Parameter sweeps in phases 3 and 4 • Status: actively used in production Phase 1 Phase 2

  14. MosGrid Science Gateway for Molecular Simulations • Requirement: Workflow enabled grid portal for UNICORE based on Liferay • Solution: • Adaptation of WS-PGRADE from GridSphere to Liferay • Development of UNICORE Submitter • Support for SAML • Development of ASM support • Status: • It’s prototyped and ~15 community members tested • Planned production provision from May 2011 • See details: • Previous presentation by Sandra Gesing: A Science Gateway for Molecular Simulations

  15. Result PS1&mzXML_1 Result PS1&mzXML_1 Result PS1&mzXML_2 Result PS1 Result PS2&mzXML_2 Result PS2&mzXML_1 Result PS2&mzXML_1 Result PS2 Result PS3&mzXML_2 Result PS3&mzXML_1 Result PS3&mzXML_1 Result PS3 • TPP Workflow Parameter Sets OUTPUT Result PS4&mzXML_2 Result PS4&mzXML_1 Result PS4&mzXML_1 Result PS4 PS1 ParamA = A2 ParamB = B3 ParamC = C1 PS2 ParamA = A2 ParamB = B2 ParamC = C1 INPUT Result PS5&mzXML_2 PS3 ParamA = A2 ParamB = B1 ParamC = C1 Result PS5&mzXML_1 Result PS5&mzXML_1 Result PS5 PS4 ParamA = A1 ParamB = B3 ParamC = C1 parameters.properties ParamA = A1; A2 ParamB = B1; B2; B3 ParamC = C1 combine PS5 ParamA = A1 ParamB = B2 ParamC = C1 Result PS6&mzXML_2 Result PS6&mzXML_1 Result PS6&mzXML_1 Result PS6 PS6 ParamA = A1 ParamB = B1 ParamC = C1 Data Set file_list.txt mzXML_1 mzXML_2 mzXML_3 mzXML_1 mzXML_2 mzXML_3 Step 2 Step 1 Swiss Proteomics Portal • Status: • Production since Dec 2010 in ETH Zürich • See details: • Previous presentation: Proteomics portal: an e-Science Gateway for Swiss Bioinformatics 19/05/10

  16. SHIWA VO SHIWA Science Gateway local cluster Globus DCI gLite DCI SHIWA Portal SHIWA Repository WF1 WFn Triana WE search WF GEMLCA admin Taverna WE s1 MOTEUR WE WF1 WFm s2 WE1 WEp WF list ASKALON WE Kepler WE edit WF GEMLCA Repository s2 PGRADE Workflow editor ASKALON WE MOTEUR WE GWES WE s4 WE + WF e-scientist s3 PGRADEWorkflow engine GEMLCA with GIB pre-deployed-WEs s5 invoke WE GEMLCA Service SHIWA Simulation Platform v1 (P-GRADE) and v2 (WS-PGRADE) Creating multi-workflow applications and executing them in the multi-middleware SHIWA VO

  17. Portal Access • Portal related information • www.wspgrade.hu • Access to multi-grid environment • WS-PGRADE/gUSE portal • https://guse.sztaki.hu/liferay-portal-6.0.5/ • For application porting • GASUC (Grid Appl. Support Centre) success stories • http://www.lpds.sztaki.hu/gasuc/ • Interoperable workflow and DCI support • SHIWA Simulation platform • http://www.shiwa-workflow.eu GASUC

  18. Invitation to related events • PUCOWO (P-GRADE User COmmunity WOrkshop), London, 7-8 June 2011 http://portal.p-grade.hu/pucowo2011/ • IWSG-Life (International Workshop on Science Gateways for Life Sciences), London, 8-18 June 2011 https://sites.google.com/a/staff.westminster.ac.uk/iwsg-life2011/ • DCI Summer School: Grid and Cloud for e-science Budapest, 11-17 July 2011 http://www.lpds.sztaki.hu/eudciss2011/

  19. Thank you for your attention! Questions? kacsuk@sztaki.hu www.lpds.sztaki.hu

More Related