Positioning dynamic storage caches for transient data
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Positioning Dynamic Storage Caches for Transient Data PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on
  • Presentation posted in: General

Positioning Dynamic Storage Caches for Transient Data. Sudharshan VazhkudaiOak Ridge National Lab Douglas Thain University of Notre Dame Xiaosong Ma North Carolina State Univ. Vince FreehNorth Carolina State Univ. High Performance I/O Workshop at IEEE Cluster Computing 2006.

Download Presentation

Positioning Dynamic Storage Caches for Transient Data

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Positioning dynamic storage caches for transient data

PositioningDynamic Storage Cachesfor Transient Data

Sudharshan VazhkudaiOak Ridge National Lab

Douglas ThainUniversity of Notre Dame

Xiaosong Ma North Carolina State Univ.

Vince FreehNorth Carolina State Univ.

High Performance I/O Workshop

at IEEE Cluster Computing 2006


Problem space

Problem Space

  • Data Deluge

    • Experimental facilities: SNS, LHC (PBs/yr)

    • Observatories: sky surveys, world-wide telescopes

    • Simulations from NLCF end-stations

    • Internet archives: NIH GenBank (serves 100 gigabases of sequence data)

  • Typical user access traits on large scientific data

    • Download remote datasets using favorite tools

      • FTP, GridFTP, hsi, wget

    • Shared interest among groups of researchers

      • A Bioinformatics group collectively analyze and visualize a sequence database for a few days: Locality of interest!

    • Often times, discard original datasets after interest dissipates


Existing storage models

Existing Storage Models

  • Local Disk

    • High bandwidth local access to small data.

  • Distributed File Systems and NAS

    • Medium bandwidth for dist/shared data.

  • Mass Storage ($)

    • High latency access for disaster recovery.

  • Parallel Storage ($$$)

    • High bandwidth shared access to large data with high reliability and fault tolerance.


What s missing

Private Workstations

University Cluster

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

Medium Bandwidth

High Latency

Wide Area

Networks

What’s Missing?

Computing Cluster

Computing Cluster

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

Fat

Pipe

Fat

Pipe

Parallel Storage

Mass Storage


Needed transient storage

Needed: Transient Storage

  • High bandwidth

    • Needs to be keep up with network and archive.

    • Also needs to keep up with aggressive apps. (viz?)

  • Some management control.

    • Capacity, bandwidth, locality are all limited.

    • Need some controls in order to guarantee QoS.

  • Understandable latency.

    • Keep user informed about stage-in latency.

    • Once staged, should have consistent latency.

  • Low cost.

    • Old idea: Lots of commodity disks.

    • Can we scavenge space from existing systems?

  • Reliability useful, but not crucial.


Transient storage use cases

Transient Storage: Use Cases

  • Checkpointing Large Computations

    • Don’t need to keep all forever!

  • Impedance Matching for Large Outputs

    • Evacuate CPUs, then trickle data to archive.

  • Caching Large Inputs

    • Share same data among many local users.

  • Out of Core Datasets

    • Large temporary array split across caches.


A real example grid3 osg

A Real Example: Grid3 (OSG)

Robert Gardner, et al. (102 authors)

The Grid3 Production Grid

Principles and Practice

IEEE HPDC 2004

The Grid2003 Project has deployed a multi-virtual organization, application-driven grid laboratory that has sustained for several months the production-level services required by…

ATLAS, CMS, SDSS, LIGO…


Grid2003 the details

Grid2003: The Details

The good news:

  • 27 sites with 2800 CPUs

  • 40985 CPU-days provided over 6 months

  • 10 applications with 1300 simultaneous jobs

    The bad news on ATLAS jobs:

  • 40-70 percent utilization

  • 30 percent of jobs would fail.

  • 90 percent of failures were site problems

  • Most site failures were due to disk space!


Two transient storage projects

Two Transient Storage Projects

  • Freeloader

    • Oak Ridge Natl Lab and North Carolina State U

    • Scavenge unused desktop storage.

    • Provide a large cache for archival backends.

    • Modify scientific apps slightly for direct access.

  • Tactical Storage

    • University of Notre Dame

    • Use comp. cluster storage as flexible substrate.

    • Configure subsets for distinct needs.

    • Filesystem interfaces for existing apps.


Desktop storage scavenging

Desktop Storage Scavenging?

  • FreeLoader

    • Imagine Condor for storage

      • Harness the collective storage potential of desktop workstations ~ Harnessing idle CPU cycles

    • Increased throughput due to striping

      • Split large datasets into pieces, Morsels, and stripe them across desktops

  • Scientific data trends

    • Usually write-once-read-many

    • Remote copy held elsewhere

    • Primarily sequential accesses

  • Data trends + LAN-Desktop Traits + user access patterns make collaborative caches using storage scavenging a viable alternative!


Properties of desktop machines

Properties of Desktop Machines

  • Desktop Capabilities better than ever before

  • Space usage to Available storage ratio is significantly low in academic and industry settings

  • Increasing numbers of workstations online most of the time

    • At ORNL-CSMD, ~ 600 machines are estimated to be online at any given time

    • At NCSU, > 90% availability of 500 machines

  • Well-connected, secure LAN settings

    • A high-speed LAN connection can stream data faster than local disk I/O


Freeloader environment

FreeLoader Environment


Freeloader architecture

FreeLoader Architecture

  • Lightweight UDP

  • Scavenger device: metadata bitmaps, morsel organization

  • Morsel service layer

  • Monitoring and Impact control

  • Global free space management

  • Metadata management

  • Soft-state registrations

  • Data placement

  • Cache management

  • Profiling


Comparing freeloader with other storage systems

Comparing FreeLoader with other storage systems


Tactical storage systems tss

Tactical Storage Systems (TSS)

  • A TSS allows any node to serve as a file server or as a file system client.

  • All components can be deployed without special privileges – but with security.

  • Users can build up complex structures.

    • Filesystems, databases, caches, ...

    • Admins need not know/care about larger structures.

  • Two Independent Concepts:

    • Resources – The raw storage to be used.

    • Abstractions – The organization of storage.


Positioning dynamic storage caches for transient data

App

file transfer

App

Parrot

Simple

Filesystem

Distributed Filesystem Abstraction

Parrot

Distributed Database Abstraction

file

server

file

server

file

server

file

server

file

server

file

server

file

server

3PT

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

Cluster administrator controls

policy on all storage in cluster

Workstations owners control

policy on each machine.

App

Parrot

???

file

system

file

system

file

system

file

system

file

system

file

system

file

system


Applications high bw access to astrophys data

Disk

Disk

Disk

Disk

Disk

Disk

Applications:High BW Access to Astrophys Data

tcsh, cp, vi, emacs, fortran...

Disk

Disk

Disk

CPU

CPU

CPU

Adapter

Disk

Disk

Disk

GBs/

Day

CPU

CPU

CPU

10 TB

Logical

Volume

Scratch

Disk

Disk

Disk

Disk

CPU

CPU

CPU

GBs / Day

Disk

Disk

Disk

GBs/

Day

CPU

CPU

CPU

General Purpose Computing Cluster

Tape Archive


Applications high bw access to biometric data

Disk

Disk

Disk

CPU

CPU

CPU

Disk

Disk

Disk

Disk

Disk

CPU

CPU

CPU

Disk

Disk

Disk

Disk

Disk

CPU

CPU

CPU

Disk

Disk

Disk

Disk

Disk

CPU

CPU

CPU

General Purpose Computing Cluster

Applications:High BW Access to Biometric Data

Job

NFS I/O

Gb Ethernet

Job

Storage Archive

NFS I/O

Disk

Disk

Disk

Job

Job

NFS I/O

Job


Applications high bw access to biometric data1

Job

Disk

Disk

Job

Disk

Disk

Job

Job

Disk

Disk

Job

Applications:High BW Access to Biometric Data

Disk

Disk

Disk

Disk

Disk

CPU

CPU

CPU

Gb Ethernet

Disk

Disk

Disk

Disk

CPU

CPU

CPU

Storage Archive

Controlled

Replication

Disk

Disk

Disk

Disk

Disk

Disk

CPU

CPU

CPU

Disk

Disk

Disk

Disk

CPU

CPU

CPU

General Purpose Computing Cluster


Open problems

Open Problems

  • Combining Technologies

    • A filesystem interface for Freeloader.

    • Making TSS harness FL benefactors.

  • Seamless Data Migration

    • Not easy to move between parallel systems!

    • Can transient storage “match impedance?”

  • Performance Adaptation

    • Many axes: BW, Latency, Locality, Mgmt.

    • Can we have a system that allows for a more continuous tradeoff or reconfiguration?


Take home message

Take-Home Message

Big, fast storage archives are important, but...

Making transient storage usable, accessible, and high performance is critical to improving the end-user experience.


  • Login