non archive storage futures
Skip this Video
Download Presentation
( non-archive) Storage Futures

Loading in 2 Seconds...

play fullscreen
1 / 5

( non-archive) Storage Futures - PowerPoint PPT Presentation

  • Uploaded on

( non-archive) Storage Futures. Wahid Bhimji SRM; FTS3; xrootd ; DPM collaborations; cluster filesystems. SRM ; FTS3; xrootd. SRM is currently required on all WLCG storage It has limitations; not much of the spec is used Some ( eg . CERN!) are talking about not using it

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '( non-archive) Storage Futures' - rune

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
non archive storage futures

(non-archive)Storage Futures

Wahid Bhimji

SRM; FTS3; xrootd; DPM collaborations; cluster filesystems

srm fts3 xrootd
SRM ; FTS3; xrootd
  • SRM is currently required on all WLCG storage
    • It has limitations; not much of the spec is used
    • Some (eg. CERN!) are talking about not using it
    • There is a WLCG WG to monitor alternatives (ensure interoperation; limit proliferation; etc.)
    • BUT ATLAS and LHCb require developmentto get away from SRM and some issues are not solved So..
    • Storage for coming years needs a stable SRM interface.
    • In future it may not – there will be an interface of some sort – but it will be lighter (I hope).
  • FTS already supports gridftp-only endpoints and FTS3 will also offer http and xrootd.
  • Xrootd use is expanding
    • Big interest is “federated” storage –failover and “anydata anywhere”
    • (Other solutions e.g http can offer this and are not hep specific)
    • CMS is asking all sites to have xrootd interface by end of year
    • ATLAS is also pushing deployment – but use cases not clear…
dpm collaboration
DPM collaboration
  • DPM support at CERN decreasing from current (v. good) level
  • CERN asking for collaborators to continue to maintain DPM
  • They say they will provide “minimal” support even without collaboration (bug fixes etc.)
  • Collaboration also has advantages in terms of getting needed developments
  • On the other hand – landscapes change:
    • dCache is maybe easier to use than before; StoRM maybe more stable; Lustre and HDFS are well established
    • Next years shutdown _may_ also be an opportunity to try something different
    • Though DPM also offering “DMLite” ontop of Lustre/HDFS
options and issues my twist on a doc from jens
Options and issues:my twist on a doc from Jens
  • Join collaboration (~1FTE or 2 x 0.5)
    • Do we have the skills for core development?
    • Does DPM have a long term (support) future?
    • Is the shutdown a chance to move to something “better” (e.g. for “hotfiles”)
  • Move to something else
    • dCache; Storm/Lustre; DMLite/Lustre; DMLite/HDFS
    • Migrating data (for ATLAS a recopy is fine but there is bound to be some hassle)
    • Migrating storage (onto something new / unfamiliar) lot of work – especially for smaller sites.
    • Have a lot of DPM experience (e.g. tuning) so alternative may not work out “better” for us
  • Need to try dpm development to see how easy it is (e.g. with DMLite)
  • Need criteria if comparing alternatives. E.g:
    • Transition effort
    • Maintenance effort

For our use cases:

    • Stability
    • Functionality
    • Performance (inc. ease of tuning)
  • Both of these take time (i.e. six months evaluating could be spent training in DPM)
  • Site-admin view should have high weight...