the gsi mass storage for experiment data n.
Download
Skip this Video
Download Presentation
The GSI Mass Storage for Experiment Data

Loading in 2 Seconds...

play fullscreen
1 / 30

The GSI Mass Storage for Experiment Data - PowerPoint PPT Presentation


  • 103 Views
  • Uploaded on

The GSI Mass Storage for Experiment Data. DVEE-Palaver GSI Darmstadt Feb. 15, 2005 Horst Göringer, GSI Darmstadt H.Goeringer@gsi.de. Overview. different views current status last enhancements: - write cache - on-line connection to DAQ future plans conclusions.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The GSI Mass Storage for Experiment Data' - brett-case


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the gsi mass storage for experiment data

The GSI Mass Storage for Experiment Data

DVEE-Palaver GSI Darmstadt

Feb. 15, 2005

Horst Göringer, GSI Darmstadt

H.Goeringer@gsi.de

overview
Overview
  • different views
  • current status
  • last enhancements:

- write cache

- on-line connection to DAQ

  • future plans
  • conclusions

GSI DVEE Palaver 15.2.2005

gsi mass storage system
GSI Mass Storage System

Gsi mass STORagE system

gstore

GSI DVEE Palaver 15.2.2005

gstore storage view
gstore: storage view

GSI DVEE Palaver 15.2.2005

gstore hardware view
gstore: hardware view

3 automatic tape libraries (ATL):

(1) IBM 3494 (AIX)

8 tape drives IBM 3590 (14 MByte/s)

ca. 2300 volumes (47 TByte, 13 TByte backup)

1 data mover (adsmsv1)

access via adsmcli, RFIO read

read cache1.1 TByte

StagePool, RetrievePool

GSI DVEE Palaver 15.2.2005

gstore hardware view1
gstore: hardware view

(2) StorageTek L700 (Windows 2000)

8 tape drives LTO2 ULTRIUM (35 MByte/s)

ca 170 volumes (32 TByte)

8 data mover (gsidmxx), connected via SAN

access via tsmcli, RFIO

read cache 2.5 TByte

StagePool, RetrievePool

write cache

ArchivePool: 0.28 TByte

DAQPool: 0.28 TByte

GSI DVEE Palaver 15.2.2005

gstore hardware view2
gstore: hardware view

(3) StorageTek L700 (Windows 2000)

4 tape drives LTO1 ULTRIUM (15 MByte/s)

ca. 80 volumes (10 TByte):

backup copy of 'irrecoverable' archives ...raw

mainly for backup of user data (~ 30 TByte)

GSI DVEE Palaver 15.2.2005

gstore software view
gstore: software view

2 major components:

  • TSM (Tivoli Storage Manager) commercial

handles tape drives and robots

data base

  • GSI software (~ 80,000 lines of code)

C, sockets, threads

- interface to user (tsmcli / adsmcli, RFIO)

- interface to TSM (TSM API client)

- cache administration

GSI DVEE Palaver 15.2.2005

gstore user view tsmcli
gstore user view: tsmcli

tsmcli subcommands:

archive file* archive path

retrieve file* archive path

query file* archive path*

stage file* archive path

delete file archive path

ws_query file* archive path

pool_query pool*

*: any combination of wildcard characters (*,?) allowed

soon: file may contain list of files (with wildcard chars)

GSI DVEE Palaver 15.2.2005

gstore user view rfio
gstore user view: RFIO

rfio_[f]open

rfio_[f]read

rfio_[f]write

rfio_[f]close

rfio_[f]stat

rfio_lseek

GSI extensions (for on-line DAQ connection):

rfio_[f]endfile

rfio_[f]newfile

GSI DVEE Palaver 15.2.2005

gstore server view query
gstore server view: query

GSI DVEE Palaver 15.2.2005

gstore server view archive to cache
gstore server view: archive to cache

GSI DVEE Palaver 15.2.2005

gstore server view archive from cache
gstore server view: archive from cache

GSI DVEE Palaver 15.2.2005

gstore server view retrieve from tape
gstore server view: retrieve from tape

GSI DVEE Palaver 15.2.2005

server view retrieve from write cache
server view: retrieve from write cache

GSI DVEE Palaver 15.2.2005

gstore overall server view
gstore: overall server view

GSI DVEE Palaver 15.2.2005

server view gstore design concepts
server view: gstore design concepts
  • strict separation of control and data flow
  • no bottleneck for data
  • scalable in

capacity (tape and disk)

I/O bandwidth

  • hardwareindependent

(as long as TSM support)

  • platformindependent
  • uniquename space

GSI DVEE Palaver 15.2.2005

server view cache administration
server view: cache administration
  • multithreaded servers for read and write cache
  • each with own metadata DB
  • main tasks:

- lock/unlock files

- select data movers and file systems

- collect actual infos on

disk space

soon: data mover and disk load -> load balancing

- trigger asynchronous archiving

- disk cleaning

  • several disk pools with different attributes:

StagePool, RetrievePool, ArchivePool, DAQPool, ...

GSI DVEE Palaver 15.2.2005

usage profile batch farm
usage profile: batch farm

batch farm: ~120 double processor nodes

=> highly parallel mass storage access (read and write)

  • read requests:

'good' user: stage all files before

use wildcard chars

'bad' user: read lots of single files from tape

'bad' system: stage disk/DM crashes during analysis

  • write requests:

via write cache

distribute as uniformly as possible

GSI DVEE Palaver 15.2.2005

usage profile experiment daq
usage profile: experiment DAQ
  • several continous data streams from DAQ
  • keep same DM during life time of data stream
  • only via RFIO
  • GSI extensions necessary:

rfio_[f]endfile, rfio_[f]newfile

  • disks faster emptied than filled:

network -> disk: ~10 MByte/s

disk -> tape: ~30 MByte/s

=> time to stage for on-line analysis

  • enough disk buffer necessary for case of problems

(robot, TSM, ...)

GSI DVEE Palaver 15.2.2005

current plans new hardware
current plans: new hardware

more and safer disks:

  • write cache: all RAID

4 TByte (ArchivePool, DAQPool)

  • read cache: +7.5 TByte new RAID

StagePool, RetrievePool,

new pools, e.g. with longer file life time

  • 5 new data movers:

new fail-safe entry server

  • hosts query server, cache administration servers

-> query performance!

  • take-over in case of host failure
  • metadata DBs mirrored on 2nd host

GSI DVEE Palaver 15.2.2005

current plans merge tsmcli adsmcli
current plans: merge tsmcli /adsmcli

new command gstore:

  • replaces tsmcli and adsmcli
  • unique name space (already available)
  • users need not care in which robot data reside
  • new archive: policy computing center

GSI DVEE Palaver 15.2.2005

brief excursion future of ibm 3494
brief excursion: future of IBM 3494?
  • still heavily used
  • rather full
  • hardware highly reliable
  • should be decided this year!

GSI DVEE Palaver 15.2.2005

usage ibm 3494 aix
usage IBM 3494 (AIX)

GSI DVEE Palaver 15.2.2005

brief excursion future of ibm 34941
brief excursion: future of IBM 3494?

2 extreme options (and more in between):

  • no more money investment

use as long as possible

in a few years: move data to other robot

  • upgrade tape drives and connect to SAN

3590 (~30 GB, 14 MB/s) -> 3592 (300 GB, 40 MB/s)

new media: => 700 TByte capacity

access with available data movers via SAN

new fail-safe TSM server(Linux?)

GSI DVEE Palaver 15.2.2005

current plans load balancing
current plans: load balancing
  • acquire actual infoon no. of read/writeprocesses

for each disk, data mover, pool

  • new write request:

select resource with lowest load

  • new read request:

avoid 'hot spots'

-> create additional instances of stage file

  • new option '-randomize' for stage/retrieve

distribute equally to different data movers / disks

split into n (parallel) jobs

GSI DVEE Palaver 15.2.2005

current plans new org of dms
current plans: new org. of DMs
  • Linux platform

more familar environment (shell scripts, Unix commands, ...)

case sensitive file names

current mainstream OS for experiment DV

  • '2nd level' data movers

no SANconnection

disks filled via ('1st level') DMs with SAN connection

for stage pools with guaranteed life time of files

GSI DVEE Palaver 15.2.2005

current plans new org of dms1
current plans: new org. of DMs
  • integration of selected group file servers

as '2nd level' data movers

disk space (logically) reserved for owners

pool policy according to owners

many advantages:

no NFS => much faster I/O

files physically distributed over several servers

load balancing of gstore

disk cleaning

disadvantages:

only for exp. data, access via gstore interface

GSI DVEE Palaver 15.2.2005

current plans user interface
current plans: user interface
  • a large number of user requests:

- longer file names

- option to rename files

- more specific return codes

- ...

  • program code consolidation
  • improved error recovery after HW failures
  • support for successor of alien
  • GRID support

- gstore as Storage Element (SE)

- Storage Resource Manager (SRM)

-> new functionalities, e.g. reserve resources

GSI DVEE Palaver 15.2.2005

conclusions
Conclusions
  • GSI concept for mass storage successfully verified
  • hardware and platform independent
  • scalable in capacity and bandwidth to keep up with

- requirements of future batch farm(s)

- data rates of future experiments

  • gstore able to manage very different usage profiles
  • but still a lot of work ...

to fully reach all discussed plans

GSI DVEE Palaver 15.2.2005