Belle gfarm grid experiment at sc04
Download
1 / 19

SC04 - PowerPoint PPT Presentation


  • 318 Views
  • Updated On :

APAN Workshop Jan 27, 2005 Bangkok. Belle/Gfarm Grid Experiment at SC04. Osamu Tatebe Grid Technology Research Center, AIST. Goal and feature of Grid Datafarm. Goal Dependable data sharing among multiple organizations High-speed data access, High-performance data computing Grid Datafarm

Related searches for SC04

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' SC04' - Donna


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Belle gfarm grid experiment at sc04 l.jpg

APAN Workshop

Jan 27, 2005

Bangkok

Belle/Gfarm Grid Experiment at SC04

Osamu Tatebe

Grid Technology Research Center, AIST


Goal and feature of grid datafarm l.jpg
Goal and feature of Grid Datafarm

  • Goal

    • Dependable data sharing among multiple organizations

    • High-speed data access, High-performance data computing

  • Grid Datafarm

    • Gfarm File System– Global dependable virtual file system

      • Federates scratch disks in PCs

    • Parallel & distributed data computing

      • Associates Computational Grid with Data Grid

  • Features

    • Secured based on Grid Security Infrastructure

    • Scalable depending on data size and usage scenarios

    • Data location transparent data access

    • Automatic and transparent replica selection for fault tolerance

    • High-performance data access and computing by accessing multiple dispersed storages in parallel (file affinity scheduling)


Grid datafarm 1 gfarm file system world wide virtual file system ccgrid 2002 l.jpg

/gfarm

ggf

jp

file1

file2

aist

gtrc

file2

file1

file3

file4

Grid Datafarm (1): Gfarm file system - World-wide virtual file system [CCGrid 2002]

  • Transparent access to dispersed file data via global namespace

    • Files can be stored somewhere in a Grid

    • Applications can access Gfarm file system without any modificationas if it were mounted at /gfarm

    • Automatic and transparent replica selection for fault tolerance and access-concentration avoidance

Globalnamespace

mapping

Gfarm File System

File replica creation


Grid datafarm 2 high performance data access and computing support ccgrid 2002 l.jpg
Grid Datafarm (2): High-performance data access and computing support [CCGrid 2002]

User’s view

Physical execution view

User A submits that accesses

is allocated on a node that has

File A

File A

Job A

Job A

User B submits that accesses

is allocated on a node that has

File B

File B

Job B

Job B

network

Computational Grid

CPU

CPU

CPU

CPU

Gfarm File System

Compute and file system nodes

Shared network file system

Do not separate Storage and CPU

Scalable file I/O by exploiting local I/O


Gfarm tm data grid middleware l.jpg
Gfarm computing support [CCGrid 2002]TM Data Grid middleware

  • Open source development

    • GfarmTM version 1.0.4-4 released on Jan 11th, 2005 (http://datafarm.apgrid.org/)

      • Read-write mode support, more support for existing binary applications

    • A shared file system in a cluster or a grid

    • Accessibility from legacy applications without any modification

    • Standard protocol support by scp, GridFTP server, samba server, . . .

Metadata server

application

gfmd

slapd

* Existing applications

can access Gfarm file system

without any modification using LD_PRELOAD

Gfarm client library

CPU

CPU

CPU

CPU

gfsd

gfsd

gfsd

gfsd

. . .

Compute and file system nodes


Demonstration l.jpg
Demonstration computing support [CCGrid 2002]

  • File manipulation

    • cd, ls, cp, mv, cat, . . .

    • grep

  • Gfarm command

    • File replica creation, node & process information

  • Remote (parallel) program execution

    • gfrun prog args . . .

    • gfrun -N #procs prog args . . .

    • gfrun -G filename prog args . . .


Belle gfarm grid experiment at sc2004 l.jpg

Belle/Gfarm Grid experiment computing support [CCGrid 2002]at SC2004

1. Online KEKB/Belle distributed data analysis

2. KEKB/Belle large-scale data analysis

(terabyte-scale US-Japan file replication)


1 online kekb belle distributed data analysis 1 l.jpg
1. Online KEKB/Belle distributed data analysis (1) computing support [CCGrid 2002]

  • Online distributed and parallel data analysis of raw data using AIST and SDSC clusters

  • Realtime histogram and event data display at SC2004 conference hall

Raw data

10 MB/sec

SC2004

Gfarm file system

KEK

  • realtime histogram display

  • realtime event data display

192 nodes

53.75 TB

SDSC

AIST

128 nodes

3.75 TB

64 nodes50 TB

  • on demand data replication

  • distributed & parallel data analysis


1 online kekb belle distributed data analysis 2 l.jpg
1. Online KEKB/Belle distributed data analysis (2) computing support [CCGrid 2002]

  • Construct a shared network file system between Japan and US

  • Store KEKB/Belle raw data to the Gfarm file system

    • Physically, it is divided into N fragments, and stored on N different node

  • Every compute node can access it as if it were mounted at /gfarm

Raw data

10 MB/sec

SC2004

KEK

  • realtime histogram display

  • realtime event data display

Gfarm File System

SDSC

AIST

192 nodes

53.75 TB

64 nodes50 TB

128 nodes

3.75 TB


1 online kekb belle distributed data analysis 3 l.jpg
1. Online KEKB/Belle distributed data analysis (3) computing support [CCGrid 2002]

  • Basf is installed at /gfarm/~/belle

    • Install once, run everywhere

  • The raw data will be analyzed at AIST or SDSC just after it is stored

  • Analyzed data can be viewed at SC2004 in realtime

    • Histogram snapshot is generated every 5 minutes

Raw data

10 MB/sec

Computational Grid

SC2004

KEK

  • realtime histogram display

  • realtime event data display

Gfarm File System

SDSC

AIST

192 nodes

53.75 TB

64 nodes50 TB

128 nodes

3.75 TB


2 kekb belle large scale data analysis in a grid l.jpg
2. KEKB/Belle large-scale data analysis in a Grid computing support [CCGrid 2002]

  • Gfarm file system using SC conference hall and AIST F cluster

  • Assume data is stored at SC conference hall

    • Terabyte-scale mock data

  • Submit data analysis job at AIST F cluster

    • Required data is automatically transferred from SC to AIST on demand

    • Users just see a shared file system

    • Network transfer rate is measured

  • Conf 1: 8 parallel processes (2GBx8 data)

  • Conf 2: 16 parallel processes (8GBx16 data)


2 network machine configuration l.jpg

PC computing support [CCGrid 2002]

AIST

F cluster

PC

PC

PC

PC

10G (OC192)

PC

PC

PC

FCx16

1Gx8

PCx256

2TBx16

PCx8

2. Network & machine configuration

JGN2

Tsukuba WAN

SC2004

StorCloud

AIST

F cluster

JGN2 Japan-US

10Gbps

10Gbps

10Gbps

Chicago

Tokyo

Pittsburgh


Sc aist iperf x 8 l.jpg
SC→AIST (Iperf x 8) computing support [CCGrid 2002]

7,925,607,155 bps (Wed Nov 10 17:13:22 JST 2004)

(5-sec average bandwidth, 991 Mbps / TCP flow)


Iperf measurement l.jpg
Iperf measurement computing support [CCGrid 2002]

  • Standard TCP (Linux 2.4)

    • Socket buffer size and txqueuelen

  • No kernel patch, no TCP modification

  • No traffic shaping

  • No bottleneck, no problem


Conf 1 8 processes 2gbx8 l.jpg
Conf 1: 8 processes (2GBx8) computing support [CCGrid 2002]

2,084,209,307 bps (Fri Nov 12 03:41:54 JST 2004)

(5-sec average, 261 Mbps / TCP flow, ~disk performance of F cluster)


Conf 2 16 processes 8gbx16 l.jpg
Conf 2: 16 processes (8GBx16) computing support [CCGrid 2002]

738,920,649 bps (Fri Nov 12 05:30:35 JST 2004)

(5-sec average, 46 Mbps!! / TCP flow, ?????)


Conf 2 network traffic of jgn2 int l link l.jpg
Conf 2: network traffic of JGN2 int computing support [CCGrid 2002]’l link

Heavy traffic when application started

Heavy packet loss→ssthresh decreases


Summary l.jpg
Summary computing support [CCGrid 2002]

  • Belle/Gfarm Grid experiment at SC2004

    • 1. Online KEKB/Belle distributed data analysis

    • 2. KEKB/Belle large-scale data analysis

  • We succeeded distributed & parallel data analysis of KEKB/Belle data and realtime display at SC conference hall


Development status and future plan l.jpg
Development Status and Future Plan computing support [CCGrid 2002]

  • Gfarm – Grid file system

    • Global virtual file system

      • A dependable network shared file system in a cluster or a grid

    • High performance data computing support

      • Associates Computational Grid with Data Grid

  • Gfarm v1 Data Grid middleware

    • Version 1.0.4-4 released on Jan 11, 2005 (http://datafarm.apgrid.org/)

    • Existing programs can access Gfarm file system as if it were mounted at /gfarm

  • Gfarm v2 – towards *true* global virtual file system

    • POSIX compliant - supports read-write mode, advisory file locking, . . .

    • Performance and Robustness improved, Security enhanced.

    • Can be substituted for NFS, AFS, . . .

  • Application area

    • Scientific application (High energy physics, Astronomic data analysis, Bioinformatics, Computational Chemistry, Computational Physics, . . .)

    • Business application (Dependable data computing in eGovernment and eCommerce, . . .)

    • Other applications that needs dependable file sharing among several organizations

  • Standardization effort with GGF Grid File System WG (GFS-WG)

https://datafarm.apgrid.org/


ad