Optimizing Large-Scale Distributed Storage: XORing Elephants and ZigZag Codes

1. 236601 - Coding and Algorithms for MemoriesLecture 12

2. Array Codes and Distributed Storage

3. Large Scale Storage Systems • Big Data Players: Facebook, Amazon, Google, Yahoo,… Cluster of machines running Hadoop at Yahoo! (Source: Yahoo!) • Failures are the norm 3

4. Node failures at Facebook Date XORingElephants: Novel Erasure Codes for Big Data M. Sathiamoorthy, M. Asteris, D. Papailiopoulos, A. G. Dimakis, R. Vadali, S. Chen, and D. Borthakur, VLDB 2013 4

5. Problem Setup • Disks are stored together in a group (rack) • Disk failures should be supported • Requirements: • Support as many disk failures as possible • And yet… • Optimal and fast recovery • Low complexity

6. Problem Setup • Question 1: How many extra disks are required to support a singledisk failure? • Question 2: How many extra disks are required to support twodisk failures? • Question 3: How many extra disks are required to support d disk failures? {(x1,x2,x3,x4): x1+x2+x3+x4= 0 } A B C A+B+C {(x1,x2,x3,x4): H1∙(x1,x2,x3,x4)T=0} H1 = (1,1,1,1) {(x1,x2,x3,x4,x5): x1+x2+x3+x4=0x1+x2+x3+x5=0 } A B C A+B+C A+B+C {(x1,x2,x3,x4,x5): H2∙(x1,x2,x3,x4,x5)T=0} H2= (1,1,1,1,0; ,,,0,1) {(x1,x2,x3,x4,x5,x6): x1+x2+x3+x4=0x1+x2+x3+x5=0’x1+’x2+’x3+x6=0} A B C A+B+C A+B+C ’A+’B+’C {(x1,x2,x3,x4,x5,x6):H3∙(x1,x2,x3,x4,x5,x6)T=0} H3= (1,1,1,1,0,0; ,,,0,1,0; ’,’,’,0,1,0)

7. Reed Solomon Codes • A code with parity check matrix of the form Where is a primitive element at some extension field and O() > n-1 Claim: Every sub-matrix of size dxd has full rank

8. Reed Solomon Codes • Advantages: • Support the maximum number of disk failures • Are very comment in practice and have relatively efficient encoding/decoding schemes • Disadvantages • Require to work over large fields • Need to read all the disks in order to recover even a single disk failure – not efficient rebuild

9. Reed Solomon Codes • Advantages: • Support the maximum number of disk failures • Are very comment in practice and have relatively efficient encoding/decoding schemes • Disadvantages • Require to work over large fieldsSolution: EvenOdd Codes • Need to read all the disks in order to recover even a single disk failure – not efficient rebuildSolution: ZigZag Codes

10. EVENODD Codes • Designed by Mario Balum, Jim Brady, JehoshuaBruck, and Jai Menon • Goal: Construct array codes correcting 2 disk failures using only binary XOR operations • No need for calculations over extension fields • Code construction: • Every disk is a column • The array size is (m-1)x(m+2), m is prime • The last two arrays are used for parity

11. EVENODD Codes

12. The Repair Problem RS code • Facebook’s storage Scheme: • 10 data blocks • 4 parity blocks • Can tolerate any four disk failures 1 2 3 4 5 6 7 8 9 10 P1 P2 P3 P4 • A disk is lost – Repair job starts • Access, read, and transmit data of disks! • Overuse of system resources during single repair • Goal: Reduce repair cost in a single disk repair 12

13. ZigZag Codes • Designed by ItzhakTamo, Zhiying Wang, and JehoshuaBruck • The goal: construct codes correcting the max number of erasures and yet allow efficient reconstruction if only a single drive fails

14. ZigZag Codes • Example

15. ZigZag Codes • Lower bound: The min amount of data required to be read to recover a single drive failure • (n,k) code: n drives, k information, and n-k redundancy • M- size of a single drive in bits • For (n,n-2) code it is required to read at least 1/2 from the remaining drives, that is at least (1/2)(n-1)M bits • The last example is optimal • In general, for (n,n-r) code it required to read at least 1/r from the remaining drives (1/r)(n-1)M

16. ZigZag Codes • Example

17. ZigZag Codes • Example