pegasus planning for execution in grids
Download
Skip this Video
Download Presentation
Pegasus: Planning for Execution in Grids

Loading in 2 Seconds...

play fullscreen
1 / 17

Pegasus: Planning for Execution in Grids - PowerPoint PPT Presentation


  • 112 Views
  • Uploaded on

Pegasus: Planning for Execution in Grids. Ewa Deelman, Carl Kesselman, Gaurang Mehta, Gurmeet Singh, Karan Vahi Information Sciences Institute University of Southern California. Pegasus Acknowledgement.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Pegasus: Planning for Execution in Grids' - harrison-glass


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
pegasus planning for execution in grids

Pegasus: Planning for Execution in Grids

Ewa Deelman, Carl Kesselman, Gaurang Mehta, Gurmeet Singh, Karan Vahi

Information Sciences Institute

University of Southern California

pegasus acknowledgement
Pegasus Acknowledgement
  • Ewa Deelman, Carl Kesselman, Saurabh Khurana, Gaurang Mehta, Sonal Patil, Gurmeet Singh, Mei-Hui Su, Karan Vahi (ISI)
  • James Blythe, Yolanda Gil (ISI)
  • http://pegasus.isi.edu
  • Research funded as part of the NSF GriPhyN, NVO and SCEC projects.
grid applications
Grid Applications
  • Increasing in the level of complexity
  • Use of individual application components
  • Reuse of individual intermediate data products
  • Description of Data Products using Metadata Attributes
  • Execution environment is complex and very dynamic
    • Resources come and go
    • Data is replicated
    • Components can be found at various locations or staged in on demand
  • Separation between
    • the application description
    • the actual execution description
slide4

Abstract Workflow Generation

Concrete

Workflow Generation

pegasus planning for execution in grids1
Pegasus:Planning for Execution in Grids
  • Maps from abstract to concrete workflow
    • Algorithmic and AI based techniques
  • Automatically locates physical locations for both components (transformations) and data
    • Use Globus RLS and the Transformation Catalog
  • Finds appropriate resources to execute
    • via Globus MDS
  • Reuses existing data products where applicable
  • Publishes newly derived data products
    • Chimera virtual data catalog
slide6

Chimera is developed at ANL

By I. Foster, M. Wilde, and J. Voeckler

ligo scientific collaboration
LIGO Scientific Collaboration
  • Continuous gravitational waves are expected to be produced by a variety of celestial objects
  • Only a small fraction of potential sources are known
  • Need to perform blind searches, scanning the regions of the sky where we have no a priori information of the presence of a source
    • Wide area, wide frequency searches
  • Search is performed for potential sources of continuous periodic waves near the Galactic Center and the galactic core
  • The search is very compute and data intensive
  • LSC is using the occasion of SC2003 to initiate a month-long production run with science data collected during 8 weeks in the Spring of 2003
slide9

Additional resources used: Grid3 iVDGL resources

Thanks to everyone involved in standing up the tested and contributing the resources!

ligo acknowledgements
LIGO Acknowledgements
  • Bruce Allen, Scott Koranda, Brian Moe, Xavier Siemens, University of Wisconsin Milwaukee, USA
  • Stuart Anderson, Kent Blackburn, Albert Lazzarini, Dan Kozak, Hari Pulapaka, Peter Shawhan, Caltech, USA
  • Steffen Grunewald, Yousuke Itoh, Maria Alessandra Papa, Albert Einstein Institute, Germany
  • Many Others involved in the Testbed
  • www.ligo.caltech.edu
  • www.lsc- group.phys.uwm.edu/lscdatagrid/
  • http://pandora.aei.mpg.de/merlin/
  • LIGO Laboratory operates under NSF cooperative agreement PHY-0107417
montage
Montage
  • Montage (NASA and NVO)
    • Deliver science-grade custom mosaics on demand
    • Produce mosaics from a wide range of data sources (possibly in different spectra)
    • User-specified parameters of projection, coordinates, size, rotation and spatial sampling.

Mosaic created by Pegasus based Montage from a run of the M101 galaxy images on the Teragrid.

montage acknowledgments
Montage Acknowledgments
  • Bruce Berriman, John Good, Anastasia Laity, Caltech/IPAC
  • Joseph C. Jacob, Daniel S. Katz, JPL
  • http://montage.ipac. caltech.edu/
  • Testbed for Montage: Condor pools at USC/ISI, UW Madison, and Teragrid resources at NCSA, PSC, and SDSC.

Montage is funded by the National Aeronautics and Space Administration\'s Earth Science Technology Office, Computational Technologies Project, under Cooperative Agreement Number NCC5-626 between NASA and the California Institute of Technology.

just in time planning
Just In-time planning
  • Partition Abstract workflow into partial workflows
other applications using pegasus
Other Applications Using Pegasus
  • Other GriPhyN applications:
    • High-energy physics: Atlas, CMS (many)
    • Astronomy: SDSS (Fermi Lab, ANL)
  • Astronomy:
    • Galaxy Morphology (NCSA, JHU, Fermi, many others, NVO-funded)
  • Biology
    • BLAST (ANL, PDQ-funded)
  • Neuroscience
    • Tomography (SDSC, NIH-funded)
  • http://pegasus.isi.edu
  • Funding by NSF GriPhyN, NSF NVO, NIH
ad