1 / 17

A glance to the future

A glance to the future. Mike Mineter TOE-NeSC. Application Developers Course May 12-13, 2007 Manchester, UK. Overview. WMProxy GridWay RESPECT – EGEE Initiative to collect useful tools that work with gLite See EGEE application portal: http://egeena4.lal.in2p3.fr/index.php

karen-stephenson
Download Presentation

A glance to the future

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A glance to the future Mike Mineter TOE-NeSC Application Developers CourseMay 12-13, 2007 Manchester, UK

  2. Overview • WMProxy • GridWay • RESPECT – EGEE Initiative to collect useful tools that work with gLite • See EGEE application portal: http://egeena4.lal.in2p3.fr/index.php • Under construction • GridWay is one of the soon-to-be RESPECTed tools • Projects to watch include • ETICS • OMII-Europe

  3. New Functionality: WMProxy • WMProxy server • Will replace the old C++ based socket connection service • Implements an interoperable interface • Web Service based • WS-I compliant • WMProxy client • Provides C++ based WMS command-line User Interface (UI), which executes all the needed operation automatically • Provides multi language (C++, Java and Python) provided APIs

  4. LB Server Workload Manager LB Proxy WMProxy Job Controller CondorG Job Controller CondorC Log Monitor UserInterface WMS Architecture overview LCG CE gLite CE gLite WMS

  5. JDL: Single Types • Single Jobs • Normal: single and simple batch job with no peculiar requirements • MPICH: a parallel application to be run on the nodes of a cluster using the MPICH implementation of the message passing interface • new MPI flavours support planned • Interactive: a job whose standard streams are forwarded to the submitting client, that can actually interact and steer the job execution by providing real-time input information

  6. JDL: Compound Jobs • Definition • Aggregation of Single/Normal Jobs • Benefits • One Shot submission for (up to thousands of) jobs • Single call to WMProxy server • Single AuthN and AuthZ process • Submission time reduction • SingleIdentification to manage all jobs (father Job) • Not an actual Job, used to monitor the whole bunch • Sharing of files between jobs

  7. Middleware structure • Applications have access both to Higher-level Grid Services and to Foundation Grid Middleware • Higher-Level Grid Services are supposed to help the users building their computing infrastructure but should not be mandatory • Foundation Grid Middleware will be deployed on the EGEE infrastructure • Must be complete and robust • Should allow interoperation with other major grid infrastructures • Should not assume the use of Higher-Level Grid Services Applications Higher-Level Grid Services Workload Management Replica Management Visualization Workflow Grid Economies ... Foundation Grid Middleware Security model and infrastructure Computing (CE) and Storage Elements (SE) Accounting Information and Monitoring

  8. GridWay • GridWay • one of the tools recognised by EGEE’s RESPECT • Alternative to WMS • Examples of use: • Many similar jobs • Short jobs • Resources outside EGEE also to be used • User-site-specific policies are required (priorities of users’ jobs) • …

  9. Contents • What is GridWay? • Which are the benefits of using GridWay? • How do I use GridWay in EGEE? • Who is using GridWay in EGEE? • Where can I get GridWay?

  10. What is GridWay? DRMAA Results standard API (OGF DRMAA) Command Line Interface $> Applications CLI Application-Infrastructure decoupling .C, .java .C, .java open source job execution management resource brokering Grid Meta- Scheduler GridWay Basic Grid services Standard interfaces end-to-end (e.g. TCP/IP) Globus, gLite, … Grid Middleware highly dynamic & heterogeneous high fault rate PBS SGE Infrastructure

  11. Which are the benefits of using GridWay? GridWayComplements gLite Providing the Following Benefits

  12. Who is using GridWay in EGEE? Massive Ray Tracing CD-HIT workflow Fusion Biomed DRMAA interface VO Schedulers Users Users GridWay GridWay EGEE RB Services: BDII, GRAM,GridFTP EGEE Resource Broker gLite gLite gLite SGE Cluster PBS Cluster PBS Cluster

  13. Where can I get GridWay? Download the software From the Gridway webpage: www.gridway.org From the ETICS repository From the Globus CVS repository (cvs.globus.org) More Information • Gridway webpage: www.gridway.org • Application porting with GridWay http://www.gridway.org/successstories/applicationporting.php • Infrastructures using GridWay http://www.gridway.org/successstories/projectsinfrastructures.php

  14. Two projects adding value to EGEE Significant for future application developers…. • ETICS • Build/test for grid services • Spin-off from gLite development & certification • Used in OMII-Europe software repository (among others) • OMII-Europe • Creating / re-engineering services that use standards • Effect will include bridge-building across grids • The vision: a VO will be able to use services across gLite / UNICORE/ Globus/ CROWN / OMII-UK grids

  15. What will OMII-Europe do? • Initial focus on providing common interfaces and integration of major Grid software infrastructures • Common interoperable services: • Data Access, Virtual Organisation Management, Portal, Accounting, Job Submission and Job Monitoring • Capability to add additional services • Infrastructure integration • Initial EGEE/UNICORE/Globus/CROWN interoperability • Interoperable security framework

  16. B R B R B R NMI Test Config NMI Build Config OMII-Europe Repository and ETICS PUBLIC VIEW Project Repository ETICS (Build & Test) Software Repository (CVS, Subversion, tar.gz, zip) P P S S NMI Build Config Build Artefact Repository (rpm, deb, tar.gz, zip) NMI Scripts Condor Pools Created Artefact

  17. Summary • Application developers will benefit from up-coming functionality from: • gLite with WMProxy • RESPECT – prominent is GridWay • Related projects • ETICS: build and test of grid services • OMII-Europe: components that will permit a VO’s resources to span grids

More Related