the kangaroo approach to data movement on the grid l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Kangaroo Approach to Data Movement on the Grid PowerPoint Presentation
Download Presentation
The Kangaroo Approach to Data Movement on the Grid

Loading in 2 Seconds...

play fullscreen
1 / 11

The Kangaroo Approach to Data Movement on the Grid - PowerPoint PPT Presentation


  • 600 Views
  • Uploaded on

The Kangaroo Approach to Data Movement on the Grid Author: D. Thain, J. Basney, S.-C. Son, and M. Livny From: HPDC 2001 Presenter: NClab, KAIST, Hyonik Lee Motivation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Kangaroo Approach to Data Movement on the Grid' - PamelaLan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the kangaroo approach to data movement on the grid

The Kangaroo Approach to Data Movement on the Grid

Author: D. Thain, J. Basney, S.-C. Son, and M. Livny

From: HPDC 2001

Presenter: NClab, KAIST, Hyonik Lee

motivation
Motivation
  • In many Grid applications, client usually gets data from data server, processes them and sends the results back to data server.
problem
Problem
  • While data are being transferred to destination server, error or latency can occur due to
    • Server crash
    • Performance variation of server
    • Exhausted resources

-> notresilient

  • The throughput of grid application is not good because the client should be blocked until the data are transferred in such a hostile environment.
solution approach
Solution Approach
  • Use intermediary memory or disk buffer.
    • Background processes move data and handling errors.
  • Interface
    • Get
    • Put
    • Commit
    • push
advantages
Advantages
  • One-hop Kangaroo
    • Insulates the client form many difficulties
      • Network failure, destination machine crash
      • Exhausted resources (e.g. BW)
    • Read operation can be satisfied from cached data.
  • Multi-hop Kangaroo
    • Multiple hops help avoid the need to co-allocate network resources along all hops.
    • Multiple hops increase the available spooling space.
architecture

File

System

Architecture

Application

open, read, write, close, fsync

Adaptation

get, put, push, commit

open, read, write, close, fsync

Consistency

Consistency

Consistency

get, put, commit, push

get, put,

push,commit

ack

ack

Kangaroo

Transport

Kangaroo

Transport

Kangaroo

Transport

TCP/IP

TCP/IP

TCP/IP

adaptation layer
Adaptation Layer
  • It is unreasonable to expect programmers to convert existing applications to work with Kangaroo.
  • Adaptation Layer converts standard POSIX operations into Kangaroo operations.
  • A number of operations are missing in the Kangaroo interface, but it is sufficient to admit many grid applications that simply must read and write data.

read

write

fsync

exit

open without create

open with create

lseek

close

get

put

push

commit

Kangaroo

POSIX

Just update table

consistency layer
Consistency Layer
  • Commit causes the caller to block until all outstanding changes have been written to some stable storage.
  • Push causes the caller to block until all outstanding changes have been delivered to their respective destinations.
kangaroo transport layer
Kangaroo Transport Layer
  • Performance
    • Uses all available resources (net, memory, disk) to maximize throughput
  • Error management
    • Retry
    • Delay
slide11

Blocking procedure call

Non-blocking message