storage information and the site status board n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Storage information and the Site Status Board PowerPoint Presentation
Download Presentation
Storage information and the Site Status Board

Loading in 2 Seconds...

play fullscreen
1 / 11

Storage information and the Site Status Board - PowerPoint PPT Presentation


  • 67 Views
  • Uploaded on

Storage information and the Site Status Board. Andrea Sciabà. MTF meeting 6/10/201. Outline. SSB storage view Collected information Issues and ambiguities Conclusions. SSB storage view. The goal is to collect storage-related site information in a table The SSB provides historical plots

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Storage information and the Site Status Board' - tale


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
storage information and the site status board

Storage information and the Site Status Board

Andrea Sciabà

MTF meeting6/10/201

outline
Outline
  • SSB storage view
  • Collected information
  • Issues and ambiguities
  • Conclusions
ssb storage view
SSB storage view
  • The goal is to collect storage-related site information in a table
  • The SSB provides historical plots
  • Can use colour codes to highlight anomalous values
  • A previous version existed but it was not maintained and was never used
  • Useful links:
    • http://dashb-ssbcern.ch/dashboard/request.py/siteview#currentView=storage
    • https://twiki.cern.ch/twiki/bin/view/CMS/MonStorageSSB
phedex information
PhEDEx information
  • Custodial: amount of custodial data at the site
  • Non-custodial: amount of non-custodial data at the site
  • Incoming: amount of subscribed data not at the site
  • Total subscribed or resident?
  • By group?
    • Caveat: proliferation of columns
pledges
Pledges
  • Taken from SiteDB
    • Yellow: > 1 quarter old
    • Red: > 1 year old
  • Will add those from REBUS
  • Question:
    • Which ones are to be trusted more?
bdii information
BDII information
  • These GLUE attributes are used and so defined by WLCG:
    • TotalOnlineSize: the total online space available at a given moment
    • UsedOnlineSize: the space occupied by available and accessible files that are not candidates for garbage collection
    • FreeOnlineSize: Total – Used
    • InstalledOnlineCapacity: used for accounting purposes, the size of the physical space of a SE
    • Same for Nearline
bdii issues and ambiguities
BDII issues and ambiguities
  • Information clearly wrong for all T1 sites apart from ASGC
  • What to report if a SE creates more replicas of some files?
two conflicting interpretations
Two conflicting interpretations
  • “Site” interpretation
    • Used is the space occupied by files
    • Free is the space not occupied by files
  • “User” interpretation
    • Used is the amount of data stored
    • Free is the amount of data that could be added
  • We need both! Naively, one could:
    • Ask for the BDII to provide the “site” info
    • Use the PhEDEx agent to calculate the “user” info
      • Although the “free” space could be tricky: maybe add the “overhëad” as a site-specific parameter?
pragmatic approach
Pragmatic approach
  • For the time being, try to make the best possible use of the BDII information
    • Fix the obvious issues (e.g. RAL declares 2.2 Exabytes of used tape)
    • Maybe in 90% of cases the numbers are usable for accounting and operations…
  • Or, ask at least the T1’s to provide their storage usage numbers in a standard format
    • And stop asking them to update the weekly twiki
    • Publish the numbers in the SSB
conclusions
Conclusions
  • The SSB historical plots could be used to generate storage accounting plots
  • The SSB is not convenient for too fine-grained information
    • E.g. disk usage by group
  • Three main sources of information:
    • Pledges: REBUS or SiteDB?
    • PhEDEx agent and namespace dumps for space usage (from CMS point of view)
    • Site information for physical space usage (either from BDII or from custom sources)