Peck transparent distributed backup using chirp
1 / 11

Peck: Transparent Distributed Backup Using Chirp - PowerPoint PPT Presentation

  • Uploaded on

Peck: Transparent Distributed Backup Using Chirp. Graduate Operating Systems, Fall 2005. Matthew Van Antwerp. December 15, 2005. Outline. Existing Methods Strengths and Weaknesses Chirp Overview Peck: Storage and Retrieval Mapfiles Conclusion. Existing Backup Methods.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Peck: Transparent Distributed Backup Using Chirp' - ovidio

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Peck transparent distributed backup using chirp
Peck: Transparent Distributed Backup Using Chirp

Graduate Operating Systems, Fall 2005

Matthew Van Antwerp

December 15, 2005


  • Existing Methods

  • Strengths and Weaknesses

  • Chirp Overview

  • Peck: Storage and Retrieval

  • Mapfiles

  • Conclusion

Existing Backup Methods

  • Dedicated Backup Server

  • Portable Media (CD, DVD, etc.)

  • Freenet

  • USB Thumbdrives

  • Chirp

Chirp: Distributed Storage Pool at ND

  • Composed mostly of department and lab computers - catalog list

  • Each system (about 200 in total) offers up spare hard drive space.

  • Can be accessed through libchirp API, command line tools, or parrot.

  • Peck sits on top of Chirp.


Peck Structure









Peck Function

  • Input file list

  • Attempts upload and download of test file to each server to learn permissions

  • Locates sufficient servers for upload

  • Writes filename and server name to mapfile for each uploaded file

  • Uploads copy of mapfile to multiple servers

Peck Mapfile

  • Hypothetical: Your hard drive crashes and you lose all your data (yes, it will happen to you one day).

  • Luckily you have been backing up your files via Peck.

  • Peck scours the servers for your mapfile (relatively slow due to no knowledge of Chirp servers).

  • When it finds a copy, it retrieves the file and opens it.

  • One by one, Peck retrieves files listed in mapfile from the relevant servers.

Peck Mapfile Versions

  • How do we know we are retrieving the proper mapfile when we upload new files or retrieve files?

  • Also, how can we keep from filling up Chirp servers with redundant copies?

  • Answer: through careful use of mapfiles.

  • After first run of peck, we have a mapfile on the servers which we retrieve on next run.

  • We check mapfile for a given filename when attempting to upload and update appropriately.

  • Before uploading new mapfile, delete all old mapfiles.


  • Checking permissions: slow (18.15s avg)

  • Upload: similar to ftp times

  • Mapfile retrieval after total system loss: slow

  • Download (after mapfile retrieved): similar to ftp times.

  • Updating mapfile and removing old mapfiles: fairly slow.

  • Very little overhead besides the obvious bottlenecks.


  • Peck achieves many of the strengths of other backup methods while avoiding their weaknesses.

  • Easy to use – simply give the application a list of files to upload (easily used in conjunction with file age scripts). An ideal cron job.

  • Cheap (free), fairly efficient (although not largely scalable), easy to setup and maintain, and transparent.

  • All necessary information for retrieval is stored on the Chirp servers.