the kangaroo approach to data movement on the grid l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Kangaroo approach to Data movement on the Grid PowerPoint Presentation
Download Presentation
The Kangaroo approach to Data movement on the Grid

Loading in 2 Seconds...

play fullscreen
1 / 35

The Kangaroo approach to Data movement on the Grid - PowerPoint PPT Presentation


  • 528 Views
  • Uploaded on

The Kangaroo approach to Data movement on the Grid Rajesh Rajamani (raj@cs.wisc.edu) June 03, 2002 Condor and the Grid What is the Grid? Grid applications tend to have demanding I/O needs. Process/produce large amounts of data Need for an efficient mechanism to transfer data Motivation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Kangaroo approach to Data movement on the Grid' - Jeffrey


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the kangaroo approach to data movement on the grid
The Kangaroo approach to Data movement on the Grid

Rajesh Rajamani (raj@cs.wisc.edu)

June 03, 2002

condor and the grid
Condor and the Grid
  • What is the Grid?
  • Grid applications tend to have demanding I/O needs. Process/produce large amounts of data
  • Need for an efficient mechanism to transfer data
motivation
Motivation
  • Suppose you want to transfer 10 large (10s of GB) and 100 small files (in KB) from UW to Vietnam -
  • Would you use FTP, if you know that -
    • Network connection to Vietnam is minimal at the best of times ?
    • Would you be satisfied with the reliability and performance of FTP?
motivation4

File

System

Motivation
  • Shortcomings of FTP -

1. Performance suffers in case of transient errors

Fast, but

unreliable

link

slow

link

D

App

R

R

R

Router

R

Destination

D

Data

motivation5
Motivation
  • TCP checksum can fail to detect errors for large xfers
    • TCP uses 16-bit checksum, which is one’s complement of the one’s complement sum of the TCP segment (RFC1071)

0110011001100110

0101010101010101

0000111100001111

------------------------

1100101011001010 0011010100110101

tcp checksum
TCP checksum
  • All burst errors of 16 or fewer bytes will be detected except when 0x0000 is replaced by 0xFFFF and vice versa
  • Other errors are detected only probabilistically

1110011001100110

0101010101010101 Original checksum =

1000111100001111 0011010100110101

------------------------

1100101011001011 (one’s complement sum)

0011010100110100 (checksum)

checksum errors
Checksum errors

0110011001100110

0011010101010101 Original checksum =

0010111100001111 0011010100110101

------------------------

1100101011001010(one’s complement sum)

0011010100110101 (checksum)

  • Error goes undetected!!!
motivation contd
Motivation (contd.)

3. Doesn’t support persistent queues

4. Opens a connection for every transfer (which in most cases is a single file)

motivation contd9

0

0

1

1

.

.

.

.

Calculate (n-k)

FEC blocks

.

.

.

.

.

.

.

.

N -1

K -1

K

N

K

File to be transferred

N >> K

FEC Block

Motivation (contd.)
  • No support to efficiently broadcast/multicast files
the vision

App

File

System

File

System

File

System

File

System

The Vision

K

=Kangaroo server

K

K

K

Data Movement

System

K

K

K

K

outline
Outline
  • Motivation
  • Architecture
  • Replication
  • Conclusions
architecture
Architecture
  • Message based architecture
  • Layers
    • Application
    • Adaptation
    • Consistency
    • Transport
architecture13

Blocking procedure call

Non-blocking message

File

System

Architecture

Application

open, read, write, close, fsync

Adaptation

get, put, push, commit

open, read, write, close, fsync

Consistency

Consistency

get, put, commit, route

get, put,

push,commit

ack

ack

Kangaroo

Transport

Kangaroo

Transport

Kangaroo

Transport

TCP/IP

TCP/IP

TCP/IP

adaptation layer

Blocking procedure call

Non-blocking message

Adaptation layer

Application

open, read, write, close, fsync

Adaptation

get, put, push, commit

Consistency

interposition agents
Interposition agents

Application

Application

Std Library

Agent

Kernel

Std Library

Kernel

slide16

Bypass and PFS

  • Bypass - General-purpose tool for trapping and redirecting standard library procedures.
  • PFS – Pluggable File System
  • PFS traps all I/O operations. It converts file-related operations to Kangaroo operations.
  • Can be applied at run-time to any dynamically-linked program:
    • setenv LD_PRELOAD ./pfs_agent.so
    • vi /kangaroo/antipholus.cs.wisc.edu/u/r/a/raj/.cshrc
    • gcc /http/www.cs.wisc.edu/~raj/example.c -o /kangaroo/antipholus.cs.wisc.edu/u/r/a/raj/temp/example
bypass mechanism
Bypass Mechanism
  • A static call is bound to a function definition when the library is linked. Dynamic calls are bound at run time by the loader.
  • The preload/bypass mechanism is a hook only into the dynamic loader
multiple bypass
Multiple Bypass

Application

Global DS

Entry

points

open _open __open read write

layer

Switch

opendir readdir socket accept connect

Interposition agent

printf scanf open _open __open read write

Standard Library

printf calls write
printf() calls write()

Application

Global DS

Entry

points

write open _open __open read

layer

Switch

opendir readdir socket accept connect

Interposition agent

printf scanf open _open __open read write

Standard Library

architecture20

Blocking procedure call

Non-blocking message

File

System

Architecture

Application

open, read, write, close, fsync

Adaptation

get, put, push, commit

open, read, write, close, fsync

Consistency

Consistency

get, put, commit, route

get, put,

push,commit

ack

ack

Kangaroo

Transport

Kangaroo

Transport

Kangaroo

Transport

TCP/IP

TCP/IP

TCP/IP

consistency layer

Blocking procedure call

Non-blocking message

File

System

Consistency layer

Adaptation

get, put, push, commit

open, read, write, close, fsync

Consistency

Consistency

get, put, commit, route

get, put,

push,commit

ack

ack

Kangaroo

Transport

Kangaroo

Transport

kangaroo message format
Kangaroo message format

Source IP

Destination IP

File name

Operation – put, get, push

Data/Offset

consistency layer23
Consistency layer
  • Interface
    • Get block, put block, commit blocks, push blocks
  • Semantics
    • Ordering - Order preserving
    • Reliability - Detects success
    • Duplication - Delivers at most once
ordering and duplication
Ordering and Duplication
  • For most applications, order of writes doesn’t matter. E.g. - writes to different offsets
  • Only renames and writes that overlap are non-idempotent. Majority of the operations are idempotent
  • Consistency layer deals with non-idempotent operations by using kangaroo_push
architecture25

Blocking procedure call

Non-blocking message

File

System

Architecture

Application

open, read, write, close, fsync

Adaptation

get, put, push, commit

open, read, write, close, fsync

Consistency

Consistency

get, put, commit, route

get, put,

push,commit

ack

ack

Kangaroo

Transport

Kangaroo

Transport

Kangaroo

Transport

TCP/IP

TCP/IP

TCP/IP

transport layer
Transport layer

Consistency

Consistency

get, put, commit, route

get, put,

push,commit

ack

ack

Kangaroo

Transport

Kangaroo

Transport

Kangaroo

Transport

TCP/IP

TCP/IP

TCP/IP

transport layer27
Transport layer
  • Interface
    • Send message, query route, query status
  • Semantics
    • Ordering - None (or worse!)
    • Reliability - Likely, but not guaranteed.
    • Duplication - Unlikely, but possible.
  • Performance
    • Uses all available resources (net, memory, disk) to maximize throughput
  • Replication necessary to provide reliability
    • Must cache blocks until delivery is acknowledged
    • Replication may introduce inconsistencies
slide28

Cedar – Bandwidth regulator

  • Standard socket abstraction.
  • Enforces limits on how much bandwidth can be consumed across multiple times scales.
  • Also measures congestion and reports to locally-determined manager.
  • Example:
    • If conditions are good, do not exceed 10Mb/s.
    • If there is competition for the link, fall back to no more than 1Mb/s.
replication
Replication
  • Endpoints keep copies of data blocks until they are acknowledged.
  • Receiver sends cumulative acks and commits unbroken sequences.
  • Replication can be costly
  • Choices -
    • Replicate @ sender
    • Pass the buck - Change end-point
slide30

D

D

D

sender

S

D

Data

Pass the Buck

ACK

  • Delete the local copy after a one-hop ack. Requires atomic accept and sync. (Similar to email)

K

K

K

K

S

D

conclusions
Conclusions
  • Large scale file transfer is not a solved problem. Many interesting research issues
  • Kangaroo provides an efficient mechanism to transfer large datasets over the WAN. It does so by using all available resources (net, memory, disk) to maximize throughput
  • Kangaroo is a data movement system, an overlay network, a distributed buffer system
kangaroo
Kangaroo
  • Uses three basic techniques to address some issues
    • Uses a virtual circuit to transfer data
    • Buffering on intermediate nodes improves performance
    • Retries till it succeeds (when it encounters an error).
as side effects
As side effects
  • Kangaroo
    • Allows applications to hide latency by overlapping I/O and CPU
    • Maximizes use of resources

However,

    • No support for inter-process consistency
    • Does NOT provide low-latency single file transfers
    • Does not use checksums, but will do so in the near future
    • No support for multicasting to multiple clients – AS YET
future multiroute and multicast

App

File

System

File

System

File

System

File

System

Future - Multiroute and Multicast

K

K

K

K

K

K

K

references
References
  • D. Thain, J. Basney, S.-C. Son, and M. Livny, "The Kangaroo Approach to Data Movement on the Grid", Tenth IEEE Symposium on High Performance Distributed Computing (HPDC10), San Francisco, California, August 7-9, 2001
  • J.Stone and Craig Partridge, “When The CRC and TCP Checksum Disagree, SIGCOMM 2000
  • Gemmell, Jim, Schooler, Eve, and Gray, Jim, Fcast Multicast File Distribution, IEEE Network, Vol. 14, No. 1, Jan 2001