slide1
Download
Skip this Video
Download Presentation
SRB Services

Loading in 2 Seconds...

play fullscreen
1 / 21

SRB Services - PowerPoint PPT Presentation


  • 86 Views
  • Uploaded on

SRB Services. Introduction. An Overview of SRB CCLRC and SRB Case Study: e-Minerals Mini-Grid SRB Production Services Questions. Managing Data. Historically data has been STORED rather than MANAGED Problems arising from this include: Scaling Distribution

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SRB Services' - tova


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
introduction
Introduction
  • An Overview of SRB
  • CCLRC and SRB
  • Case Study: e-Minerals Mini-Grid
  • SRB Production Services
  • Questions
managing data
Managing Data
  • Historically data has been STORED rather than MANAGED
  • Problems arising from this include:
    • Scaling
    • Distribution
    • Access Control, Authentication, Security
    • Data Migration
    • Data Curation
what is srb
What is SRB?
  • Storage Resource Broker (SRB) is a software product developed by the San Diego Supercomputing Centre (SDSC).
  • Allows users to access files and database objects across a distributed environment.
  • Actual physical location and way the data is stored is abstracted from the user
  • Allows the user to add user defined metadata describing the scientific content of the information
how srb works
MCAT

Database

c

d

MCAT

Server

b

e

f

SRB A

Server

SRB B

Server

g

a

SRB

Client

How SRB Works
  • 4 major components:
    • The Metadata Catalogue (MCAT)
    • The MCAT-EnabledSRB Server
    • The SRB Storage Server
    • The SRB Client
the mcat database
The MCAT Database
  • The MCAT database is a metadata repository that provides a mechanism for storing information used by the SRB system.
  • Includes both
    • Internal system data required for running the system
    • Application (user) metadata regarding data sets being brokered by SRB.
the mcat server
The MCAT Server
  • At least one SRB Server must be installed on the node that can access the MCAT database. This is known as the MCAT-Enabled SRB Server.
  • MCAT SRB Server works directly against the MCAT database to provide SRB Services
  • All other SRB Servers interact through the MCAT Server
the srb server
The SRB Server
  • The SRB Server is a middleware application that accepts requests from clients and obtains/queries/manages the necessary data sets.
  • It queries the MCAT SRB Server to gather information on datasets and supplies this back to the SRB client.
srb client tools
SRB Client Tools
  • Provide a user interface to send requests to the SRB server.
  • 4 main interfaces:
    • Command line (S-Commands)
    • MS Windows (InQ)
    • Web based (MySRB).
    • Java (JARGON)
  • Web Services (MATRIX)
concepts
Concepts
  • Location: A physical node running an SRB Server
  • Physical Resource: A storage area managed by an SRB Server
  • Logical Resource: One or more Physical Resources – can be distributed
  • Collection – Data abstraction of resources
srb in detail
C, C++,

Linux I/O

Unix

Shell

SRB

Databases

DB2, Oracle,

PostgreSQL

Archives

ADS, HPSS,

ADSM,DMF

File Systems

Unix, NT,

Mac OSX

SRB in Detail

Application

Resource,

User

Java, NT

Browsers

Prolog

Python

Third-party

copy

Web

User

Defined

Remote

Proxies

MCAT

HRM

DataCutter

Application

Meta-data

administration
Administration
  • Users / Locations / Resources must be managed
  • Two methods for doing this:
    • Java MCAT Admin Tool
    • Command line tools
cclrc and srb
CCLRC and SRB
  • The Data Management Group in CCLRC started working with SRB in November 2002 after a fact finding mission to the USA.
  • There was an immediate requirement for a storage based product that allowed the addition of searchable metadata
  • Generated lots of internal interest, which led to a number of projects with SRB
srb example cms
SRB Example: CMS
  • Largest project using CCLRC SRB services to date is the CERN CMS experiment.
  • SRB chosen for ‘Pre-Challenge Production’, producing data for Data Challenge 2003/2004 (DC03/DC04)
  • Need to prove data can be transferred, replicated and stored at LHC rates
  • DC04 provided key input to SRB Version 3.2
srb case study e minerals
SRB Case Study: e-Minerals
  • UK e-science project for modelling the atomistic processes involved in environmental issues
e minerals requirements
e-Minerals Requirements
  • Data Management Requirements
    • Scientists want to store input and output files from simulations in different locations
    • manage their own files/data via the web
    • give access to other project members
    • give temporary access to others
architecture
Cambridge

Reading

SRB Resource

Bath

SRB Resource

SRB Resource

Architecture

Web Browser

DaresburyApp Server

Application server runs SRB software

MySRB

Eminerals MiniGrid

UK Grid

SRB Server

Oracle Client

Database server holds locations of files

Daresbury Database server

MCAT

building on experience new services
Building on Experience - New Services
  • CCLRC SRB Service
    • Initial service availability: October
    • Proposed Customers include:
      • ISIS Facility, British Atmospheric Data Centre (BADC), AHDS
    • ADS interface (with Containers)
    • Test systems in place
  • NGS SRB Service
    • e-Minerals, e-Materials, Integrative Biology
srb services
SRB Services
  • SRB version 3.2
    • Performance, scalability and reliability
  • Ongoing Service Enhancements
    • Automatic failover
    • Product Documentation and Training - Collaboration with SDSC
summary
Summary
  • Links established with SRB community and SDSC
  • Real SRB projects implemented
  • Creating new generation of SRB Production Systems
  • Can help community with:
    • SRB Test Systems
    • SRB Production Systems
    • SRB Training and Support
  • Contributing to future versions
ad