The mass storage system at jlab today and tomorrow
Download
1 / 19

The Mass Storage System at JLAB - Today and Tomorrow - PowerPoint PPT Presentation


  • 84 Views
  • Uploaded on

The Mass Storage System at JLAB - Today and Tomorrow. Andy Kowalski. From CLAS DAQ. From Hall A,C DAQ. 100 mbit. 1000 mbit. FCAL (100MByte). SCSI. DST/Cache File Servers. Jefferson Lab Farm and Mass Storage Systems 2000. Tape Servers. Cache File Servers. DB Server. Work File Servers.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' The Mass Storage System at JLAB - Today and Tomorrow' - tulia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

From CLAS DAQ

From Hall A,C DAQ

100 mbit

1000 mbit

FCAL (100MByte)

SCSI

DST/Cache File Servers

Jefferson Lab

Farm and Mass Storage Systems

2000

Tape Servers

Cache File Servers

DB Server

Work File Servers

Batch and Interactive Farm


Farm

  • Total Batch Farm 6172.6 SPECint95

    • 125 - Dual Processor Linux Systems

    • 4 - Dual Processor Solaris Systems

  • 5 Interactive Farm Systems

    • 1 - Quad Processor Linux System

    • 2 – Dual Processor Linux Systems

    • 2 – Quad Processor Solaris Systems


Disk storage
Disk Storage

  • MetaStor (Work) File Servers – Qty. 2

    • 256MB Memory

    • 100mbit Ethernet

    • Ultra Wide SCSI RAID Systems

    • RAID-5

    • Disk Space – 4TB

      • FS3/FS4 - 1TB

      • FS5/FS6 - 3TB

    • Serving NFS

    • User Managed Disk Space


Disk storage cont
Disk Storage cont.

  • Linux (Cache) File Servers – Qty. 9

    • Dual 650/700MHz Systems

    • 512MB ECC Memory

    • Gigabit Ethernet

    • Mylex PCI RAID controller

    • RAID-0

    • Disk Space – 5.8TB

      • Qty. 4 with 400GB

      • Qty. 5 with 876GB

    • Linux (Red Hat 6.2) - 2.2.18pre11-va1.1smp

    • Serving NFS

    • Automatically Managed Disk Space


Tape storage
Tape Storage

  • StorageTek PowderHorn 9310

    • 350 tape mounts per hour

    • Holds 6000 tapes

  • Redwood Tape Drives – Qty. 8

    • 50GB tape capacity

    • 10MB/second

  • 9840 Tape Drives – Qty. 10

    • 20GB tape capacity

    • 10MB/second


Tape storage cont
Tape Storage cont.

  • Open Storage Manager

    • Computer Associates dropped support January 2000

    • Not distributed

    • Two installations – mss1 and mss2

  • TapeServer Front-End

    • Written in Java

    • Makes two OSM servers act as one

    • Stages data to/from disk from/to tape

    • A user interface


From CLAS DAQ

From Hall A,C DAQ

100 mbit

1000 mbit

FCAL (100MByte)

SCSI

DST/Cache File Servers

Jefferson Lab

Farm and Mass Storage Systems

2001

Tape Servers

Cache File Servers

DB Server

Work File Servers

Batch and Interactive Farm


Projects
Projects

  • JASMine

    • Replacement for OSM

    • Distributed Data Movers and Cache Managers

    • Scalable to the needs of the experiments

    • Smart scheduling

    • Off-site cache or ftp servers for data exporting


Projects cont
Projects cont.

  • File Exports

    • JASMine Cache Software

      • GRID Aware - PPDG

    • FTP

    • BBFTP

    • WEB

    • Tape


Jasmine logical storage organization
JASMine Logical Storage Organization

  • Store

    • A logical entity made up of libraries, servers, data movers, and a database.

  • Storage Group

    • An object that belongs to a store and is itself a collection of storage groups or volume sets.

  • Volume Set

    • An object that belongs to a storage group and is itself a collection of volumes.

  • Volume

    • A unit of storage media.

  • Bitfile

    • The copy of a file that has been copied into a store.


Store

Storage

Group 1

Storage

Group 2

Storage

Group 3

Volume

Set 1

Volume

Set 2

Volume

Set 3

Storage

Group 4

Storage

Group 5

Volume

Set 4

Volume

Set 5

Vol

Vol

Vol

Bitfiles

Bitfiles

Bitfiles

Vol

Vol

Bitfiles

Bitfiles


Jasmine physical storage organization
JASMine Physical Storage Organization

  • Store

    • A logical entity made up of libraries, servers, data movers, and a database.

  • Library

    • A set of volumes and drives.

  • Drive

    • A media reader or writer.

  • Volume

    • A unit of storage media.


Jasmine services
JASMine Services

  • Request Manager

    • Handles user requests and queries.

  • Scheduler

    • Prioritizes user requests for tape access.

      • priority = share / ( .01 + (num_a * ACTIVE_WEIGHT) + (num_c * COMPLETED_WEIGHT) )

  • Log Manager

    • Writes out log and error files and databases.

    • Sends out notices for failures.

  • Library Manager

    • Mount and dismounts tapes as well as other library related tasks.


Jasmine services 2
JASMine Services 2

  • Data Mover

    • Dispatcher

      • Keeps track of available local resources and starts requests the local system can work on.

    • Cache Manager

      • Manages a disk or disks for pre-staging data to and from tape.

      • Sends and receives data to and from clients.

    • Volume Manager

      • Manages tapes for availability.

    • Drive Manager

      • Manages tape drives for usage.


Request

Manager

Database

Scheduler

Request

Manager

Log

Manager

Dispatcher

Client

Cache

Manager

Volume

Manager

Drive

Manager

Library

Manager

Library

Manager

Disk

Drive

Data Mover


What to expect in 2001
What to Expect in 2001

  • Work File Servers

    • Additional 5 TB

    • Linux File Servers??

  • Cache File Servers

    • Additional 5-10TB


2001 cont
2001 cont.

  • Tape Storage

    • JASMine Installed

    • 5-10 Data Movers

    • 10 Additional Tape Drives (9840 or 9940)



ad