1 / 47

Gathering at the Well: Creating Communities for Grid I/O

Gathering at the Well: Creating Communities for Grid I/O. New framework needed. Remote I/O is possible anywhere Build notion of locality into system? What are possibilities? Move job to data Move data to job Allow job to access data remotely Need framework to expose these policies .

issac
Download Presentation

Gathering at the Well: Creating Communities for Grid I/O

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Gathering at the Well:Creating Communities for Grid I/O

  2. New framework needed • Remote I/O is possible anywhere • Build notion of locality into system? • What are possibilities? • Move job to data • Move data to job • Allow job to access data remotely • Need framework to expose these policies

  3. Key elements • Storage appliance, interposition agents, schedulers and match-makers • Mechanism not policies • Policies are exposed to an upper layer • We will however demonstrate the strength of this mechanism

  4. To infinity and beyond • Speedups of 2.5x possible when we are able to use locality intelligently • This will continue to be important • Data sets are getting larger and larger • There will always be bottlenecks

  5. Outline • Motivation • Components • Expressing locality • Experiments • Conclusion

  6. I/O communities • Mechanism which allow either • jobs to move to data, or • data to move to jobs, or • data to be accessed remotely • Framework to evaluate these policies

  7. Grocers, butchers, cops • Members of an I/O community • Storage appliances • Interposition agents • Scheduling systems • Discovery systems • Match-makers • Collection of CPU’s

  8. Storage appliances • Should run without special privilege • Flexible and easily deployable • Acceptable to nervous sys admins • Should allow multiple access modes • Low latency local accesses • High bandwidth remote puts and gets

  9. GFTP Chirp HTTP FTP Common protocol layer Control flow Data flow Dispatcher Multiple concurrencies Transfer Manager NeST Storage Manager Physical storage layer

  10. Interposition agents • Thin software layer interposed between application and OS • Allow applications to transparently interact with storage appliances • Unmodified programs can run in grid environment

  11. PFS: Pluggable File System

  12. Scheduling systems and discovery • Top level scheduler needs ability to discover diverse resources • CPU discovery • Where can a job run? • Device discovery • Where is my local storage appliance? • Replica discovery • Where can I find my data?

  13. Match-making • Match-making is the glue which brings discovery systems together • Allows participants to indirectly identify each other • i.e. can locate resources without explicitly naming them

  14. Condor and ClassAds

  15. Outline • Motivation • Components • Expressing locality • Experiments • Conclusion

  16. INFN UW I/O Communities

  17. Two I/O communities • INFN Condor pool • 236 machines, about 30 available at any one time • Wide range of machines and networks spread across Italy • Storage appliance in Bologna • 750 MIPS, 378 MB RAM

  18. Two I/O communities • UW Condor pool • ~900 machines, 100 dedicated for us • Each is 600 MIPS, 512 MB RAM • Networked on 100 Mb/s switch • One was used as a storage appliance

  19. Who Am I This Time? • We assumed the role of an Italian scientist • Database stored in Bologna • Need to run 300 instances of simulator

  20. Hmmm…

  21. Three way matching Refers to NearestStorage. Knows where NearestStorage is. Job Ad Machine Ad Storage Ad match Job Machine NeST

  22. Two way ClassAds Type = “job” TargetType = “machine” Cmd = “sim.exe” Owner = “thain” Requirements = (OpSys==“linux”) Type = “machine” TargetType = “job” OpSys = “linux” Requirements = (Owner==“thain”) Machine ClassAd Job ClassAd

  23. Type = “storage” Name = “turkey.cs.wisc.edu” HasCMSData = true CMSDataPath = /cmsdata” Storage ClassAd Three way ClassAds Type = “job” TargetType = “machine” Cmd = “sim.exe” Owner = “thain” Requirements = (OpSys==“linux”) && NearestStorage.HasCMSData Type = “machine” TargetType = “job” OpSys = “linux” Requirements = (Owner==“thain”) NearestStorage = ( Name = “turkey”) && (Type==“Storage”) Machine ClassAd Job ClassAd

  24. Outline • Motivation • Components • Expressing locality • Experiments • Conclusion

  25. BOOM!

  26. CMS simulator sample run • Purposefully choose a run with high I/O to CPU ratio • Accesses about 20 MB of data from a 300 MB database • Writes about 1 MB of output • ~160 seconds execution time • on a 600 MIPS machine with local disk

  27. Policy specification • Run only with locality • Requirements = (NearestStorage.HasCMSData) • Run in only one particular community • Requirements = (NearestStorage.Name == “nestore.bologna”) • Prefer home community first • Requirements = (NearestStorage.HasCMSData) • Rank = (NearestStorage.Name == “nestore.bologna” ) ? 10 : 0 • Arbitrarily complex • Requirements = ( NearestStorage.Name == “nestore.bologna”) || ( ClockHour < 7 ) || ( ClockHour > 18 )

  28. Policies evaluated • INFN local • UW remote • UW stage first • UW local (pre-staged) • INFN local, UW remote • INFN local, UW stage • INFN local, UW local

  29. Completion Time

  30. CPU Efficiency

  31. Conclusions • I/O communities expose locality policies • Users can increase throughput • Owners can maximize resource utilization

  32. Future work • Automation • Configuration of communities • Dynamically adjust size as load dictates • Automation • Selection of movement policy • Automation

  33. For more info • Condor • http://www.cs.wisc.edu/condor • ClassAds • http://www.cs.wisc.edu/condor/classad • PFS • http://www.cs.wisc.edu/condor/pfs • NeST • http://www.nestproject.org

  34. Local only

  35. Remote only

  36. Both local and remote

  37. I/O communities are an old idea, right? • File servers and administrative domains • No, not really. We need • more flexible boundaries • simple mechanism by which users can express I/O community relationships • hooks into system that allow users to use locality

  38. Grid applications have demanding I/O needs • Petabytes of data in tape repositories • Scheduling systems have demonstrated that there are idle CPUs • Some systems • move jobs to data • move data to jobs • allow job remote access to data • No one approach is always “best”

  39. Easy come, easy go • In a computation grid, resources are very dynamic • Programs need rich methods for finding and claiming resources • CPU discovery • Device discovery • Replica discovery

  40. Bringing it all together Distributed Repository Storage appliance Execution site CPU Discovery System Job Agent Short-haul I/O Long-haul I/O Device Discovery System Replica Discovery System

  41. Conclusions • Locality is good • Balance point between staging data and accessing it remotely is not static • depends on specific attributes of the job • data size, expected degree of re-reference, etc • depends on performance metric • CPU efficiency or job completion time

  42. Implementation • NeST • storage appliance • Pluggable File System (PFS) • interposition agent built with Bypass • Condor and ClassAds • scheduling system • discovery system • match-maker

  43. Jim Gast and Bart say ... • Too many bullet slides • Contributions • scientist doesn’t want to name bec • resources are dynamic and • name is irrelevant • hooks into system to allow users to express and take advantage of locality

  44. Jim Gast and Bart say ... • everyone knows locality is good - but there is no way to express this and run jobs on the grid • I/O communities are mechanism by which user can use locality and specify policies to optimize job performance

  45. 4 earth-shattering revelations 1) The grid is big. 2) Scientific data-sets are large. 3) Idle resources are available. 4) Locality is good.

  46. Mechanisms not policies • I/O communities are a mechanism not a policy • A higher layer is expected to choose application appropriate policies • We will however demonstrate the strength of the mechanism by defining appropriate policies for one particular application

  47. Experimental results • Implementation • Environment • Application • Measurements • Evaluation

More Related