1 / 12

GridPP2 Workshop Summary Report: Data Management Achievements

The team at Glasgow University, including William Bell, David Cameron, Tony Doyle, Gavin McCance, Paul Millar, and Caitriana Nicholson, collaborated closely with WP2 of European DataGrid. They focused on Grid optimization, simulating the Grid from a data-centric viewpoint, and developing secure metadata services for experiments.

ccarolyn
Download Presentation

GridPP2 Workshop Summary Report: Data Management Achievements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WP2: Data Management Gavin McCance University of Glasgow GridPP2 Workshop, UCL

  2. EDG Summary ReportWP2: Data Management

  3. The team • Based at Glasgow University • William Bell • David Cameron • Tony Doyle • Gavin McCance • Paul Millar • Caitriana Nicholson • We formed close collaboration with WP2 of European DataGrid • Became an integral part of the WP2 team • All our products were delivered jointly with EDG

  4. What we set out to do? • Initial work-plan was in two key areas: • Grid Optimisation • Simulating the Grid from the data-centric viewpoint • Applying the results from the simulations to develop algorithms for optimisation of replicas • Development of optimisation software for replica management • Metadata management • Development of secure metadata services for the experiments

  5. What we actually did… • Grid Optimisation • Developed a data-centric Grid simulation [OptorSim] (fully documented final release available now) • Compared of a variety of replica optimisation techniques and algorithms (many publications) • Worked on developing and refining these algorithms • Exploitation of some of the early work, we developed the Replica Optimisation Service (ROS) component of the Replica Management System • provides file access cost estimation to the user and the WP1 resource broker.

  6. What we actually did… • Metadata Management • Early production of the Spitfire browser service allowing remote access to RDBMS via a web browser • Early adoption of GSI security into Spitfire to allow secure use of the service using Grid certificates • Initial adoption of web services into EDG with the creation of the web services API version of Spitfire, allowing programmatic access to remote RDMBS • The web services technology built for Spitfire was subsequently factored out and used in all other WP2 services • The security built into Spitfire was subsequently factored out and led to the creation of the WP2 Java security modules

  7. What we actually did… • Replica Management Services • As focus of EDG changed to “stability, stability, stability” and production, we changed with it • Helped factor-out the web services and security frameworks from Spitfire • Moved into the core replica management services • Replica Manager • Replica Location Service • Replica Metadata Catalogue • Replica Optimisation Service • Provided continued support for these products to EDG and LCG • Led the thorough performance testing of our products for WP2’s final EU deliverable

  8. What we actually did… • WP2 Integration and Testing • Significant integration expertise developed • Paul Millar is the WP2 integration team member • Glasgow ran the WP2 CVS repository • We created the testing framework for WP2 software • WP2 Testbed support • Provided support for the WP2 mini-testbed • Machines managed at ScotGrid and at CERN • WP2 Management • Gavin McCance is WP2 deputy • Major contributions to WP2 deliverables

  9. What we actually did… • Standards • Contribution of GGF Data Area documents • Publications • Work was fully reported: 15 or so papers published…

  10. EDG Effort Delivered • Substantial UK effort delivered to EDG : ~150% of contracted

  11. Exploitation • Exploitation of the technology to deliver the next solutions to the experiments and to enable their own solutions • Stateful web services • Security • Service oriented • Focus on manageability and robustness – easily deployable • On the metadata side, emphasis upon Grid enabling the experiments own metadata solutions

  12. WP2: What we learned • Software development process… • sometimes learned the hard way…! • Much feedback from our users on the software • Very useful for the design process in the next projects • The amount of effort required to make production quality services out of our software • Take all this forward into GridPP2!

More Related