1 / 32

stdchk : A Checkpoint Storage System for Desktop Grid Computing

stdchk : A Checkpoint Storage System for Desktop Grid Computing. Samer Al-Kiswany – UBC. Matei Ripeanu – UBC Sudharshan S. Vazhkudai – ORNL Abdullah Gharaibeh – UBC. Oak Ridge National Laboratory. The University of British Columbia. Checkpointing Introduction.

brittneytaylor
Download Presentation

stdchk : A Checkpoint Storage System for Desktop Grid Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. stdchk: A Checkpoint Storage System for Desktop Grid Computing Samer Al-Kiswany – UBC Matei Ripeanu – UBC Sudharshan S. Vazhkudai – ORNL Abdullah Gharaibeh – UBC Oak Ridge National Laboratory The University of British Columbia

  2. Checkpointing Introduction Checkpointing uses: fault tolerance, debugging, or migration. Typically, an application running for days on hundreds of nodes (e.g. a desktop gird ) saves checkpoint images periodically. . . . C C C C ICDCS ‘08

  3. Deployment Scenario ICDCS ‘08

  4. The Challenge • Although checkpointing is necessary: • It is a pure overhead from the performance point of view. Most of the time spent writing to the storage system. • Generates a high load on the storage system Requirement: High performance, scalable, and reliable storage system optimized for checkpointing applications. Challenge: Low cost, transparent support for checkpointing at file-system level. ICDCS ‘08

  5. Checkpointing Workload Characteristics • Write intensive application ( bursty ). e.g., a job running on hundreds of nodes. periodically checkpoints 100s of GB of data. • Write once, rarely read during application execution. • Potentially high similarity between consecutive checkpoints. • Applications specific checkpoint image life span. When it is safe to delete the image ? ICDCS ‘08

  6. Why Checkpointing-Optimized Storage System? • Optimizing for checkpointing workload can bring valuable benefits: • High throughput through specialization. • Considerable storage space and network effort saving. through transparent support for incremental checkpointing. • Simplified data management by exploiting the particulaities of checkpoint usage scenarios. • Reduce the load on a share file-system • Can be built atop scavenged resources – low cost. ICDCS ‘08

  7. stdchk A checkpointing optimized storage system built using scavenged resources. ICDCS ‘08

  8. Outline • stdchk architecture • stdchk features • stdchk system evaluation ICDCS ‘08

  9. stdchk Architecture Manager (Metadata management) Benefactors (Storage nodes) Client (FS interface) ICDCS ‘08

  10. stdchk Features • High-throughput for write operations • Support for transparent incremental checkpointing • Simplified data management • High reliability through replication • POSIX file system API – as a result using stdchk does not require modifications to the application. ICDCS ‘08

  11. Optimized Write Operation Alternatives Write procedure alternatives: • Complete local write • Incremental write • Sliding window write ICDCS ‘08

  12. Optimized Write Operation Alternatives Write procedure alternatives: • Complete local write • Incremental write • Sliding window write Compute Node Application stdchk stdchk FS Interface Disk ICDCS ‘08

  13. Optimized Write Operation Alternatives Write procedure alternatives: • Complete local write • Incremental write • Sliding window write Compute Node Application stdchk stdchk FS Interface Disk ICDCS ‘08

  14. Disk Optimized Write Operation Alternatives Write procedure alternatives: • Complete local write • Incremental write • Sliding window write Compute Node Application stdchk stdchk FS Interface Memory ICDCS ‘08

  15. Write Operation Evaluation Testbed: 28 machines Each machine has : two 3.0GHz Xeon processors, 1 GB RAM, two 36.5GB SCSI disks. ICDCS ‘08

  16. Achieved Storage Bandwidth • Sliding Window write achieves high bandwidth (110 MBps) • Saturates the 1 Gbps link The average ASB over a 1 Gbps testbed. ICDCS ‘08

  17. stdchk Features • High throughput write operation • Transparent incremental checkpointing • Checkpointing optimized data management • POSIX file system interface – no required modification to the application ICDCS ‘08

  18. Transparent Incremental Checkpointing • Incremental checkpointing may bring valuable benefits: • Lower network effort. • Less storage space used. But : How much similarity is there between consecutive checkpoints ? How can we detect similarities between checkpoints? Is this fast enough? ICDCS ‘08

  19. Hashing X Checkpoint T0 X T0 Y Y Z Z Similarity Detection Mechanism – Compare-by-Hash ICDCS ‘08

  20. Hashing Checkpoint T1 W Y T1 Z W Similarity Detection Mechanism – Compare-by-Hash Will store T1 X T0 Y Z ICDCS ‘08

  21. Similarity Detection Mechanism • How to divide the file into blocks? • Fixed-size blocks + compare-by-Hash (FsCH) • Content-based blocks + compare-by-Hash (CbCH) ICDCS ‘08

  22. FsCH Insertion Problem B1 B2 B3 B4 B5 Checkpoint i B1 B2 B3 B4 B5 B6 Checkpoint i+1 Result: Lower similarity detection ratio. ICDCS ‘08

  23. offset HashValueK= 0 ? HashValueK= 0 ? HashValueK= 0 ? Content-based Compare-by-Hash (CbCH) B1 B2 B3 B4 Checkpoint i m bytes Hashing k bits ICDCS ‘08

  24. Content-based Compare-by-Hash (CbCH) B1 B2 B3 B4 Checkpoint i B1 BX B3 B4 Checkpoint i+1 Result: Higher similarity detection ratio. But: Computationally intensive. ICDCS ‘08

  25. Evaluating Similarity Between Consecutive Checkpoints The Applications : BMS* and BLAST Checkpointing interval: 1, 5 and 15 minutes * Checkpoints by Pratul Agarwal (ORNL) ICDCS ‘08

  26. Similarity Ratio and Detection Throughput The table presents the average rate of detected similarity and the throughput in MB/s (in brackets) for each heuristic. But: Using the GPU, CbCH achieves over 190 MBps throughput !! - StoreGPU: Exploiting Graphics Processing Units to Accelerate Distributed Storage Systems, S. Al-Kiswany, A. Gharaibeh, E. Santos-Neto, G. Yuan, M. Ripeanu, HPDC, 2008. ICDCS ‘08

  27. Compare-by-Hash Results FsCH slightly degrades achieved bandwidth. But reduces the storage space used and network effort by 24% Achieved Storage Bandwidth ICDCS ‘08

  28. Outline • stdchk architecture • stdchk features • stdchk overall system evaluation ICDCS ‘08

  29. Steady Nodes Leave Nodes Join stdchk Scalability • stdchk sustains high loads : • Number of nodes • Workload 7 clients: Each client writes 100 files (100MB each). Total of 70GB. stdchk pool of 20 benefactor nodes. ICDCS ‘08

  30. Experiment with Real Application Application : BLAST Execution time: > 5 days Checkpointing interval : 30s Stripe width : 4 benefactors Client machine: two 3.0GHz Xeon processors, SCSI disks. ICDCS ‘08

  31. Summary stdchk : A checkpointing optimized storage system built using scavenged resources. stdchk features: • High throughput write operation • Saves considerable disk space and network effort. • Checkpointing optimized data management • Easy to adopt – implements a POSIX file system interface • Inexpensive - built atop scavenged resources Consequently, stdchk: • Offloads the checkpointing workload from the shared FS. • Speeds up the checkpointing operations (reduces checkpointing overhead) ICDCS ‘08

  32. Thank you netsyslab.ece.ubc.ca ICDCS ‘08

More Related