Peck transparent distributed backup using chirp
This presentation is the property of its rightful owner.
Sponsored Links
1 / 11

Peck: Transparent Distributed Backup Using Chirp PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on
  • Presentation posted in: General

Peck: Transparent Distributed Backup Using Chirp. Graduate Operating Systems, Fall 2005. Matthew Van Antwerp. December 15, 2005. Outline. Existing Methods Strengths and Weaknesses Chirp Overview Peck: Storage and Retrieval Mapfiles Conclusion. Existing Backup Methods.

Download Presentation

Peck: Transparent Distributed Backup Using Chirp

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Peck transparent distributed backup using chirp

Peck: Transparent Distributed Backup Using Chirp

Graduate Operating Systems, Fall 2005

Matthew Van Antwerp

December 15, 2005


Peck transparent distributed backup using chirp

Outline

  • Existing Methods

  • Strengths and Weaknesses

  • Chirp Overview

  • Peck: Storage and Retrieval

  • Mapfiles

  • Conclusion


Peck transparent distributed backup using chirp

Existing Backup Methods

  • Dedicated Backup Server

  • Portable Media (CD, DVD, etc.)

  • Freenet

  • USB Thumbdrives

  • Chirp


Peck transparent distributed backup using chirp

Strengths and Weaknesses


Peck transparent distributed backup using chirp

Chirp: Distributed Storage Pool at ND

  • Composed mostly of department and lab computers - catalog list

  • Each system (about 200 in total) offers up spare hard drive space.

  • Can be accessed through libchirp API, command line tools, or parrot.

  • Peck sits on top of Chirp.


Peck transparent distributed backup using chirp

`

Peck Structure

Peck

libchirp

chirp

server

chirp

server

chirp

server


Peck transparent distributed backup using chirp

Peck Function

  • Input file list

  • Attempts upload and download of test file to each server to learn permissions

  • Locates sufficient servers for upload

  • Writes filename and server name to mapfile for each uploaded file

  • Uploads copy of mapfile to multiple servers


Peck transparent distributed backup using chirp

Peck Mapfile

  • Hypothetical: Your hard drive crashes and you lose all your data (yes, it will happen to you one day).

  • Luckily you have been backing up your files via Peck.

  • Peck scours the servers for your mapfile (relatively slow due to no knowledge of Chirp servers).

  • When it finds a copy, it retrieves the file and opens it.

  • One by one, Peck retrieves files listed in mapfile from the relevant servers.


Peck transparent distributed backup using chirp

Peck Mapfile Versions

  • How do we know we are retrieving the proper mapfile when we upload new files or retrieve files?

  • Also, how can we keep from filling up Chirp servers with redundant copies?

  • Answer: through careful use of mapfiles.

  • After first run of peck, we have a mapfile on the servers which we retrieve on next run.

  • We check mapfile for a given filename when attempting to upload and update appropriately.

  • Before uploading new mapfile, delete all old mapfiles.


Peck transparent distributed backup using chirp

Analysis

  • Checking permissions: slow (18.15s avg)

  • Upload: similar to ftp times

  • Mapfile retrieval after total system loss: slow

  • Download (after mapfile retrieved): similar to ftp times.

  • Updating mapfile and removing old mapfiles: fairly slow.

  • Very little overhead besides the obvious bottlenecks.


Peck transparent distributed backup using chirp

Conclusion

  • Peck achieves many of the strengths of other backup methods while avoiding their weaknesses.

  • Easy to use – simply give the application a list of files to upload (easily used in conjunction with file age scripts). An ideal cron job.

  • Cheap (free), fairly efficient (although not largely scalable), easy to setup and maintain, and transparent.

  • All necessary information for retrieval is stored on the Chirp servers.


  • Login