network coding and distributed storage n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Network Coding and Distributed Storage PowerPoint Presentation
Download Presentation
Network Coding and Distributed Storage

Loading in 2 Seconds...

play fullscreen
1 / 46

Network Coding and Distributed Storage - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Network Coding and Distributed Storage. Kenneth Shum (Joint work with Minghua Chen, Hanxu Hou and Hui Li ). Window Azure data centers. Inside a data center. http://technoblimp.com. Data distribution. Encode and distribute a data file to n storage nodes. Data File: “Butterfly”.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Network Coding and Distributed Storage' - brand


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
network coding and distributed storage

Network Coding andDistributed Storage

Kenneth Shum

(Joint work with Minghua Chen, Hanxu Hou and Hui Li)

inside a data center
Inside a data center

http://technoblimp.com

kshum

data distribution
Data distribution
  • Encode and distribute a data file to n storage nodes.

Data File: “Butterfly”

kshum

data collector
Data collector
  • Data collector can retrieve the whole file by downloading from any k storage nodes.

“Butterfly” 

kshum

three kinds of disk failures
Three kinds of disk failures
  • Transient error due to noise corruption
    • repeat the disk access request
  • Disk sector error
    • partial failure
    • detected and masked by the operating system
  • Catastrophic error
    • total failure due to disk controller for instance
    • the whole disk is regarded as erased

kshum

kshum

6

frequency of node failures
Frequency of node failures

Figure from “XORing elephants: novel erasure codes for Big Data”

by Sathiamoorthy et al.

Number of failed nodes over a single month in a 3000 node production cluster of Facebook.

kshum

7

outline of this talk
Outline of this talk
  • Comparison of three coding schemes
    • Repetition scheme
    • Traditional erasure-correcting codes
      • Reed-Solomon codes
    • Network-coding-based scheme
      • BASIC regenerating codes
  • A max-flow-min-cut bound
    • Connection to network coding.

kshum

kshum

8

distributed storage system
Distributed storage system
  • Encode a data file and distribute it to ndisks
  • (n,k) recovery property
    • The data file can be rebuilt from any kdisks.
  • Repair
    • If a node fails, we regenerate a new node by connecting and downloading data from any d surviving disks.
    • Aim at minimizing the repair bandwidth(Dimakis et al 2007).
  • A coding scheme with the above properties is called a regenerating code.

kshum

repetition scheme
Repetition scheme
  • Google File System: Replicate data 3 times
  • Gmail: Replicate data 21 times

kshum

2x r epetition scheme
2x Repetition scheme

Divide the datafile into 2 parts

1G

A

A, B

1G

Data Collector

B

1G

A

1G

Cannot toleratedouble disk failures

B

kshum

repair is easy for repetition based system

1G

Repair is easy for repetition-based system

New node

A

A

B

A

Repair bandwidth =1G

B

kshum

reed solomon code
Reed-Solomon Code

Divide the file into 2 parts

A

A, B

Data Collector

B

A+B

It can toleratedouble disk failures

A+2B

kshum

13

repair requires essentially decoding the whole file
Repair requires essentially decoding the whole file

A

A

New node

1G

B

1G

A+B

Repair bandwidth = 2G

A+2B

kshum

kshum

14

basic regeneration code

BASIC regeneration code

Binary AdditionShiftImplementableConvolutional

Divide the datafile into 4 parts

0.5G

0.5G

0.5G

0.5G

Utilization of bit-wise shift in storage was proposed byPiret and Krol (1983), andQureshi, Foh and Cai (2012).

download from nodes 1 and 2
Download from nodes 1 and 2

1G

Data Collector

0.5G

1G

0.5G

0.5G

0.5G

16

download from nodes 1 and 3
Download from nodes 1 and 3

1G

Data Collector

0.5G

0.5G

0.5G

1G

0.5G

17

download from nodes 1 and 4
Download from nodes 1 and 4

1G

Data Collector

0.5G

0.5G

0.5G

0.5G

1G

18

download from nodes 2 and 3
Download from nodes 2 and 3

1G

Data Collector

0.5G

0.5G

0.5G

1G

0.5G

19

download from nodes 2 and 4
Download from nodes 2 and 4

1G

Data Collector

0.5G

0.5G

0.5G

0.5G

1G

20

download from nodes 3 and 4
Download from nodes 3 and 4

Data Collector

0.5G

1G

0.5G

0.5G

0.5G

1G

21

zigzag decoding
Zigzag decoding

à laGollakata and Katabi (2008)

What to solvefor P1and P2.

P1

P2

P1P2

P1

P2’

P1P2’

kshum

kshum

22

repair of basic regenerating code
Repair of BASIC regenerating code

New node

XOR

Bitwise shift and XOR

Bitwise shift and XOR

kshum

repair of basic regenerating code1

Interference alignment

Repair of BASIC regenerating code

Repair bandwidth=1.5 G

Decode the blueand red packets byzigzag decoding

kshum

outline of this talk1
Outline of this talk
  • Comparison of three coding schemes
    • Repetition scheme
    • Traditional erasure-correcting codes
      • Reed-Solomon codes
    • Network-coding-based scheme
      • BASIC regenerating codes
  • A max-flow-min-cut bound
    • Connection to network coding.

kshum

kshum

26

two modes of repair
Two modes of repair
  • Exact repair
    • The content of the new node is exactly the same as the content of the failed node
  • Functional repair
    • only requires that the (n,k) recovery property is preserved.

Functional Repair

Exact Repair

kshum

information flow graph
Information flow graph

In1

Out1

In2

Out2

DataCollector

Source

In3

Out3

In4

Out4

Dimakis et al., INFOCOMM, May, 2007

kshum

information flow graph cont d
Information flow graph (cont’d)

In5

In1

Out5

Out1

In2

Out2

Source

In3

Out3

DataCollector

In4

Out4

kshum

is 1 5g repair bandwidth optimal
Is 1.5G repair bandwidth optimal?

1

1

In5

In1

Out5

Out1

1

In2

Out2

2

1

In3

Out3

DataCollector

1

In4

Out4

1 + 2  2   0.5

kshum

not necessarily the same 1 st cut
 not necessarily the same (1st cut)

1

1

In1

In6

Out1

Out6

1

1

2

In2

Out2

DataCollector

3

2

1

In3

Out3

1

1 + 1+ 2 2

In4

Out4

kshum

not necessarily the same 2 nd cut
 not necessarily the same (2nd cut)

1

1

In1

In6

Out1

Out6

1

2

1

In2

Out2

DataCollector

2

1

In3

Out3

3

1

1 + 1+ 2 2 1 + 1+ 3  2

In4

Out4

kshum

not necessarily the same 3 rd cut
 not necessarily the same (3rd cut)

1

1

In1

In6

Out1

Out6

1

1

2

In2

Out2

2

1

DataCollector

In3

Out3

3

1

1 + 1+ 2 2 1 + 1+ 3  2

1 + 2+ 3  2

 1+ 2+ 3  1.5

In4

Out4

kshum

min cut bound
Min-cut bound
  • B: total file size
  • : storage per node
  • d: total repair bandwidth
  • k: No. of connections from a data collector

For d k

d

DC

k

Dimakis et al., INFOCOMM, May, 2007

kshum

waterfilling interpretation
Waterfilling interpretation

  • Given

B, d, k, 

  • Find the minimum storage *

d

(d–1)

*

(d–k+1)

Area = B

d

k

1

2

k

DC

kshum

tradeoff between storage and bandwidth
Tradeoff between storage and bandwidth
  • B=1
  • n>16
  • k=4
  • d=16

d=16

DC

k=4

kshum

when repair bandwidth d is very large
When repair bandwidth d is very large

d

  • * = B/k
  • Any combinations of k nodes will contain B bits.

d

(d–1)

(d-1)

(d–k+1)

* = B/k

Area = B

d

k

1

2

k

DC

kshum

minimum storage regeneration msr
Minimum-storage regeneration (MSR)

d

  • B=1
  • n>16
  • k=4
  • d=16

Area = B

1

2

k

kshum

the corresponding mincut
The corresponding mincut

d

Out1

In1

DC

S

k-1

Outn

Inn

kshum

a change of slope
A change of slope

d

  • B=1
  • n>16
  • k=4
  • d=16

Area = B

1

2

k

kshum

another change of slope
Another change of slope

d

  • B=1
  • n>16
  • k=4
  • d=16

Area = B

1

2

k

kshum

minimum bandwidth regeneration mbr
Minimum-bandwidth regeneration (MBR)

d

  • B=1
  • n>16
  • k=4
  • d=16

I n f e a s i b l e

Area = B

1

2

k

kshum

storage bandwidth tradeoff curve
Storage-bandwidth tradeoff curve
  • B=1
  • n>12
  • k=10
  • d=12

Min-Bandwidth regeneration (MBR)

Min-Storage regeneration (MSR)

kshum

other performance metrics
Other performance metrics
  • Security issues
    • A compromised storage node cannot decode any part of the data file
    • A Byzantine storage node may launch pollution attack
  • Locality
    • number of nodes contacted during repair
  • Disk I/O cost
    • Number of disk accesses during the repair process

kshum

summary
Summary
  • We can reduce repair bandwidth by network coding.
    • Tradeoff between storage and repair bandwidth.
  • BASIC regenerating codes
    • A failed storage node can be repaired by simple bit-wise shift and XOR operations.
    • Small storage overhead due to shifting.

Aug 2013

kshum

kshum

45

references
References
  • Piret and Krol, MDS convolution codes, IEEE Trans. of Information Theory, 1983.
  • Dimakis, Brighten, Wainwright and Ramchandran, Network coding for distributed storage systems, INFOCOM, 2007.
  • Gollakata and Katabi, Zigzag decoding: combating hidden terminals in wireless networks, Proc. in the ACM Sigcomm, 2008.
  • Qureshi, Foh, and Cai, Optimal solution for the index coding problem using network coding over GF(2), Proc. IEEE Conf. on Sensor Mesh and Ad Hoc Comm. and Network, 2012.
  • Sung and Gong, A zigzag decodable code with MDS property for distributed storage systems, Proc. IEEE Symp. on Information Theory, 2013.
  • Hou, Shum, Chen and Li, BASIC regenerating code: binary addition and shift for exact repair, Proc. IEEE Symp. on Information Theory, 2013.

kshum