1 / 14

Data Grid: GRASP

Data Grid: GRASP. Mike Smorul <toaster@umiacs.umd.edu>. Grid Retrieval and Search Platform. Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the UMIACS GLCF. Provides a graphical interface into data grid holdings.

beulah
Download Presentation

Data Grid: GRASP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Grid: GRASP Mike Smorul <toaster@umiacs.umd.edu>

  2. Grid Retrieval and Search Platform • Based on concepts developed in the Earth Science Data Interface (ESDI) developed at the UMIACS GLCF. • Provides a graphical interface into data grid holdings. • Access to entire GLCF holdings through the Storage Resource Broker(SRB)

  3. Earth Science Data Interface

  4. ESDI Overview • Designed to allow for intuitive browsing and searching of large geospatial data sets. • Tightly integrated set of web, ftp, and file servers. (customized to GLCF) • Distributes over 7Tb of data per month • Over 27,000 Landsat scenes and 13Tb of data available for download

  5. SRB Grid Testbed (original) • Modified the SRB to hold spatial data • Contributed Informix port of the SRB (v3.2) • Linked three ESIP sites • Tested replication between GMU and UMD. • Remote registration at UNH

  6. Lessons Learned • The SRB can easily handle textual metadata. • Spatial data could be stored into extended Informix attributes, but querying was available only through the DAI • Limited SRB MCAT to Informix based systems

  7. GRASP Architecture

  8. GRASP Architecture • GRASP uses a data grid as an abstract storage repository. • Metadata in the grid is mined from the grid itself or from external sources and published into a browsable form. • Data grids may allow for platform independent metadata, but may not be optimal for access

  9. GRASP Screenshot

  10. Grid Holdings • Registered GLCF holdings • Over 338,000 registered files • 4.4Tb total size • Granular permissions on registered holdings • No need to move all data into grid, registered pre-existing holdings in place.

  11. Current data grid • Designed using newer SRB software that allows for Federated grid model. • Created using standard SRB software configuration (postgreSQL) • Large data sites can maintain independent MCATs • Administratively independent • Ability to customize data grid to site security/data requirements while maintaining compatibility across federation. • Smaller peers can register as clients of UMD

  12. Data grid overview

  13. Growing data grid • Additional MCATs can be federated • Additional SRB aware clients can be added • Remote data supplying sites can contribute data sets from their resources • Sites needing quick local access to data can replicate to local SRB setup.

  14. Questions?

More Related