1 / 34

On-the-fly Verification of Erasure-Encoded File Transfers

On-the-fly Verification of Erasure-Encoded File Transfers. Mike Freedman & Max Krohn NYU Dept of Computer Science. Downloading Large Files From P2P Networks. For large files, transfer times are much bigger than average node uptimes.

seamus
Download Presentation

On-the-fly Verification of Erasure-Encoded File Transfers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On-the-fly Verification of Erasure-Encoded File Transfers Mike Freedman & Max Krohn NYU Dept of Computer Science

  2. Downloading Large Files From P2P Networks • For large files, transfer times are much bigger than average node uptimes. • Some files are very popular: multiple sources and multiple requesting nodes. • Is it possible to have multicast, even though sources and receivers frequently enter and leave the network.

  3. Solution: Rateless Erasure Codes Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1)

  4. Solution: Rateless Erasure Codes Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1) Wants file F

  5. Mutli-Sourced Downloads Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1)

  6. Mutli-Sourced Downloads Source (S1) Source (S2) Source (S3) Source (S4) @ @ @ Receiver (R1)

  7. “Overlapping Multicast Trees” Source (S1) Source (S2) Source (S3) Source (S4) @ Receiver (R3) Receiver (R4) Receiver (R1) Receiver (R3) Receiver (R2)

  8. Resuming Truncated Downloads Source (S1) Receiver (R1) Receiver (R2)

  9. Resuming Truncated Downloads Source (S1) Receiver (R1) Receiver (R2)

  10. Resuming Truncated Downloads @ Source (S1) Receiver (R1) Receiver (R2)

  11. Threat Model KaZaa Morpheus eDonkey 2000 Gnutella

  12. Threat Model KaZaa Morpheus eDonkey 2000 Gnutella

  13. Threat Model KaZaa Morpheus eDonkey 2000 Gnutella

  14. Threat Model KaZaa Morpheus eDonkey 2000 Gnutella

  15. KaZaa Morpheus eDonkey 2000 Gnutella Bogus Data Attack

  16. KaZaa Morpheus eDonkey 2000 Gnutella Unwanted Data Attack

  17. Attacking Erasure Encoded Transfers Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1)

  18. Attacking Erasure Encoded Transfers Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1)

  19. Erasure Encoding of Files …

  20. Easily Verifiable…. …

  21. …but Not on the Fly Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1)

  22. What Happened? • R1received checkblock cfrom S4. S4claims: blah

  23. What Happened? • R1received checkblock cfrom S4. S4claims: • R1knows: But how can R1 verify c? Wouldn’t it be nice if: Not true for SHA1!

  24. What Happened? • R1received checkblock cfrom S4. S4claims: • R1knows: • But how can R1 verify c? Wouldn’t it be nice if: Not true for SHA1!

  25. What Happened? • R1received checkblock cfrom S4. S4claims: • R1knows: • But how can R1 verify c? • Wouldn’t it be nice if: Not true for SHA1!

  26. What Happened? • R1received checkblock cfrom S4. S4claims: • R1knows: • But how can R1 verify c? • Wouldn’t it be nice if: • Not true for SHA1!

  27. A Homomorphic Hashing Scheme • Assume file block size of 8kB • Pick large prime (about 1024 bits) and small prime (about 256 bits) that divides , and 256 generators of order q: • Writes the file F as matrix, elements in

  28. How To Hash • The hash of a message or check block is an element in :

  29. How To Hash • The hash of a message or check block is an element in : • The hash of the entire file is an n-element vector of the hashes of the blocks:

  30. The Only Important Slide implies that Why?

  31. How To Encode • Checkblocks are constructed using modular addition over . • To generate a checkblock, pick a set And compute

  32. How To Verify Given the correct hash: And a check block: verify that: • Note: LHS computation is expensive!

  33. Success! Source (S1) Source (S2) Source (S3) Source (S4) Receiver (R1)

  34. Analysis • Security of the hash function based on hardness of the discrete log. • Hashes are big (1/256 the size of the file), but we can apply this process recursively. • Our paper details a batched, probabilistic verification scheme that drastically reduces exponentiations. • Verifying rate is 40x faster than download rates on a T1.

More Related