1 / 24

dCache introduction Paul Millar On behalf of the dCache team. EGI-EISCAT-3D ad-hoc meeting

dCache introduction Paul Millar On behalf of the dCache team. EGI-EISCAT-3D ad-hoc meeting. High-level Overview. dCache is.

andersoncarl
Download Presentation

dCache introduction Paul Millar On behalf of the dCache team. EGI-EISCAT-3D ad-hoc meeting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. dCache introduction Paul Millar On behalf of the dCache team. EGI-EISCAT-3D ad-hoc meeting

  2. High-level Overview

  3. dCache is... • software for providing scalable, managed storage for huge amounts of data.deployed at research institutes throughout the world and used by a diverse collection of user-communities.supported through the dCache.org collaboration, which provides:regular feature releases that are maintained with subsequent bug-fix releases.Support and advice through a variety of channels.

  4. Association and funding

  5. dCache history Disk cache Grid Storage Generic Storage Cloud Storage Authen-tication

  6. What is dCache today? Core team 8 FTEs 2 FTEs Collaborations 1.5 FTEs Student mentor programme 3 students

  7. Current and future project funding

  8. dCache key features include... • Users see a single POSIX filesystem (hard- & soft-links, etc),Transparent support for tertiary (tape) storage,Scalable bandwidth,Steerable target when reading and writing,Space management,Resilience to storage node failure,Supports transparent storage device life-cycle,Hot-spot detection and mitigation,Differentiable quality of service,Pluggable authentication,...

  9. The scientific cloud vision

  10. Some details* on how dCache operates... * Some details are deliberately omitted to keep slides manageable.

  11. dCache – under the hood dcap DBMS http ftp nfs

  12. Core components when transferring

  13. Importance of redirection

  14. Guaranteeing QoS for write WRITE Write Read READ READ File on write pool (sticky) on read pool (cached; locked) on read pool (cached; locked) File on write pool (sticky) File on write pool (sticky) on read pool (cached) on read pool (cached; locked) File on write pool (sticky) on read pool (cached; locked) File on write pool (sticky) on read pool 1 (cached) on read pool 2 (cached) File on write pool (sticky) on read pool (cached)

  15. Guaranteeing QoS for tape activity FLUSH Write Read STAGE READ File on flush pool (precious) on read pool (cached) File on flush pool (precious) File on tape on flush pool (cached) on read pool (cached) File on tape on flush pool (cached) File on tape on stage pool (cached) on read pool (cached) File on tape on read pool (cached) File on tape File on tape on stage pool (sticky)

  16. Operational experience

  17. Storage at DESY • 6 dCache instances: Hera, CMS, ATLAS, Photon, “DESY” and Cloud:

  18. Comparative numbers * Photon instance accepts ~1 TiB per month as ~1x107 files. ‡ Value is peak observed bandwidth aggregate over all clients within last 7 days. † Value is peak observed open rate (either read rate or write rate) observed within last 7 days.

  19. Other dCache instances

  20. Backup slides

More Related