Pegasus planning for execution in grids
1 / 17

Pegasus: Planning for Execution in Grids - PowerPoint PPT Presentation

  • Uploaded on

Pegasus: Planning for Execution in Grids. Ewa Deelman, Carl Kesselman, Gaurang Mehta, Gurmeet Singh, Karan Vahi Information Sciences Institute University of Southern California. Pegasus Acknowledgement.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Pegasus: Planning for Execution in Grids' - harrison-glass

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Pegasus planning for execution in grids

Pegasus: Planning for Execution in Grids

Ewa Deelman, Carl Kesselman, Gaurang Mehta, Gurmeet Singh, Karan Vahi

Information Sciences Institute

University of Southern California

Pegasus acknowledgement
Pegasus Acknowledgement

  • Ewa Deelman, Carl Kesselman, Saurabh Khurana, Gaurang Mehta, Sonal Patil, Gurmeet Singh, Mei-Hui Su, Karan Vahi (ISI)

  • James Blythe, Yolanda Gil (ISI)


  • Research funded as part of the NSF GriPhyN, NVO and SCEC projects.

Grid applications
Grid Applications

  • Increasing in the level of complexity

  • Use of individual application components

  • Reuse of individual intermediate data products

  • Description of Data Products using Metadata Attributes

  • Execution environment is complex and very dynamic

    • Resources come and go

    • Data is replicated

    • Components can be found at various locations or staged in on demand

  • Separation between

    • the application description

    • the actual execution description

Abstract Workflow Generation


Workflow Generation

Pegasus planning for execution in grids1
Pegasus:Planning for Execution in Grids

  • Maps from abstract to concrete workflow

    • Algorithmic and AI based techniques

  • Automatically locates physical locations for both components (transformations) and data

    • Use Globus RLS and the Transformation Catalog

  • Finds appropriate resources to execute

    • via Globus MDS

  • Reuses existing data products where applicable

  • Publishes newly derived data products

    • Chimera virtual data catalog

Chimera is developed at ANL

By I. Foster, M. Wilde, and J. Voeckler

Ligo scientific collaboration
LIGO Scientific Collaboration

  • Continuous gravitational waves are expected to be produced by a variety of celestial objects

  • Only a small fraction of potential sources are known

  • Need to perform blind searches, scanning the regions of the sky where we have no a priori information of the presence of a source

    • Wide area, wide frequency searches

  • Search is performed for potential sources of continuous periodic waves near the Galactic Center and the galactic core

  • The search is very compute and data intensive

  • LSC is using the occasion of SC2003 to initiate a month-long production run with science data collected during 8 weeks in the Spring of 2003

Additional resources used: Grid3 iVDGL resources

Thanks to everyone involved in standing up the tested and contributing the resources!

Ligo acknowledgements
LIGO Acknowledgements

  • Bruce Allen, Scott Koranda, Brian Moe, Xavier Siemens, University of Wisconsin Milwaukee, USA

  • Stuart Anderson, Kent Blackburn, Albert Lazzarini, Dan Kozak, Hari Pulapaka, Peter Shawhan, Caltech, USA

  • Steffen Grunewald, Yousuke Itoh, Maria Alessandra Papa, Albert Einstein Institute, Germany

  • Many Others involved in the Testbed


  • www.lsc-


  • LIGO Laboratory operates under NSF cooperative agreement PHY-0107417


  • Montage (NASA and NVO)

    • Deliver science-grade custom mosaics on demand

    • Produce mosaics from a wide range of data sources (possibly in different spectra)

    • User-specified parameters of projection, coordinates, size, rotation and spatial sampling.

Mosaic created by Pegasus based Montage from a run of the M101 galaxy images on the Teragrid.

Montage acknowledgments
Montage Acknowledgments

  • Bruce Berriman, John Good, Anastasia Laity, Caltech/IPAC

  • Joseph C. Jacob, Daniel S. Katz, JPL

  • http://montage.ipac.

  • Testbed for Montage: Condor pools at USC/ISI, UW Madison, and Teragrid resources at NCSA, PSC, and SDSC.

    Montage is funded by the National Aeronautics and Space Administration's Earth Science Technology Office, Computational Technologies Project, under Cooperative Agreement Number NCC5-626 between NASA and the California Institute of Technology.

Just in time planning
Just In-time planning

  • Partition Abstract workflow into partial workflows

Other applications using pegasus
Other Applications Using Pegasus

  • Other GriPhyN applications:

    • High-energy physics: Atlas, CMS (many)

    • Astronomy: SDSS (Fermi Lab, ANL)

  • Astronomy:

    • Galaxy Morphology (NCSA, JHU, Fermi, many others, NVO-funded)

  • Biology

    • BLAST (ANL, PDQ-funded)

  • Neuroscience

    • Tomography (SDSC, NIH-funded)


  • Funding by NSF GriPhyN, NSF NVO, NIH