1 / 15

PhEDEx: a novel approach to robust Grid data management

PhEDEx: a novel approach to robust Grid data management. Tim Barrass Dave Newbold and Lassi Tuura All Hands Meeting, Nottingham, UK 22 September 2005. What is PhEDEx?. A data distribution management system

brody-wyatt
Download Presentation

PhEDEx: a novel approach to robust Grid data management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. PhEDEx: a novel approach to robust Grid data management Tim Barrass Dave Newbold and Lassi Tuura All Hands Meeting, Nottingham, UK 22 September 2005

  2. What is PhEDEx? • A data distribution management system • Used by the Compact Muon Solenoid (CMS) High Energy Physics (HEP) experiment at CERN, Geneva • Blends traditional HEP data distribution practice with more recent technologies • Grid and peer-to-peer filesharing • Scalable infrastructure for managing dataset replication • Automates low-level activity • Allows manager to work with high level dataset concepts rather than low level file operations • Technology agnostic • Overlies Grid components • Currently couples LCG, OSG, NorduGrid, standalone sites Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  3. The HEP environment • HEP collaborations are quite large • Order of 1000 collaborators, globally distributed • CMS is only one of four Large Hadron Collider (LHC) experiments being built at CERN • Typically resources are globally distributed • Resources organised in tiers of decreasing capacity • Tier 0: the detector facility • Tier 1: large regional centres • Tier 2+: smaller sites-- Universities, groups, individuals… • Raw data partitioned between sites, highly processed ready-for-analysis data available everywhere • LHC computing demands are large • Order 10 PetaBytes per year created for CMS alone • Similar order simulated • Also analysis and user data Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  4. CMS distribution use cases • Two principle use cases- push and pull of data • Raw data is pushed onto the regional centres • Simulated and analysis data is pulled to a subscribing site • Actual transfers are 3rd party- handshake between active components important, not push or pull • Maintain end-to-end multi-hop transfer state • Can only clean online buffers at detector when data safe at Tier 1 • Policy must be used to resolve these two use cases Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  5. PhEDEx design • Assume every operation is going to fail! • Keep complex functionality in discrete agents • Handover between agents minimal • Agents are persistent, autonomous, stateless, distributed • System state maintained using a modified blackboard architecture • Layered abstractions make system robust • Keep local information local where possible • Enable site administrators to maintain local infrastructure • Robust in face of most local changes • Deletion and accidental loss require attention • Draws inspiration from agent systems, “autonomic” and peer-to-peer computing Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  6. Transfer workflow overview Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  7. Production performance Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  8. Service challenge performance Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  9. Future directions • Contractual file routing • Cost-based offers for a given transfer • Peer-to-peer data location • Using Kademlia to partition replica location information • Semi-autonomy • Agents governed by many small tuning parameters • Self modify- or use more intelligent protocols? • Advanced policies for priority conflict resolution • Need to ensure that raw data is always flowing • Difficult real-time scheduling problem Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  10. Summary • PhEDEx enables dataset level replication for the CMS HEP experiment • Currently manages 200TB+ of data, globally distributed • Real life performance of 1 TB per day sustained per site • Challenge performance of over 10TB per day • Not CMS-- or indeed HEP-- specific • Well-placed to meet future challenges • Ramping up to get to O(10)PB per year • 10-100TB per day • Data starts flowing for real in the next two years Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  11. Extra information • PhEDEx and CMS • http://cms-project-phedex.web.cern.ch/cms-project-phedex/ • cms-phedex-developers@cern.ch : feel free to subscribe! • CMS Computing model http://www.gridpp.ac.uk/eb/ComputingModels/cms_computing_model.pdf • Agent frameworks • JADE http://jade.tilab.com/ • DiaMONDs http://diamonds.cacr.caltech.edu/ • FIPA http://www.fipa.org • Peer-to-peer • Kademlia http://citeseer.ist.psu.edu/529075.html • Kenosis http://sourceforge.net/projects/kenosis • Autonomic computing • http://www.research.ibm.com/autonomic/ • General agents and blackboards • Where should complexity go?http://www.cs.bath.ac.uk/~jjb/ftp/wrac01.pdf • Agents and blackboards http://dancorkill.home.comcast.net/pubs/ Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  12. Issues • Most issues fabric-related • Most low level components experimental or not production-hardened • Tools typically unreliable under load • MSS access a serious handicap • PhEDEx plays very fair, keeping within request limits and ordering requests by tape when possible • Main problem is keeping in touch with the O(3) people at each site involved in deploying fabric, administration &c Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  13. Deployment • 8 regional centres, 16 smaller sites • 110TB, replicated ~twice • 1 TB per day sustained • On standard Internet Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  14. Testing and scalability Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

  15. PhEDEx architecture Tim Barrass, Bristol, tim.barrass@bristol.ac.uk

More Related