1 / 76

On-The-Fly Verification of Rateless Erasure Codes

On-The-Fly Verification of Rateless Erasure Codes. Max Krohn (MIT CSAIL) Michael Freedman and David Mazières (NYU). Multicast Authentication: Dead/Exhausted. On-The-Fly Verification of Rateless Erasure Codes. Max Krohn (MIT CSAIL) Michael Freedman and David Mazières (NYU). The Setting.

roch
Download Presentation

On-The-Fly Verification of Rateless Erasure Codes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On-The-Fly Verification of Rateless Erasure Codes Max Krohn (MIT CSAIL) Michael Freedman and David Mazières (NYU)

  2. Multicast Authentication: Dead/Exhausted On-The-Fly Verification of Rateless Erasure Codes Max Krohn (MIT CSAIL) Michael Freedman and David Mazières (NYU)

  3. The Setting • A large file F • Linux ISO (650MB) • H(F) is available • signed by Publisher (RedHat) • A handful of untrusted sources/mirrors S1,…S8

  4. A Handful of Senders

  5. The Setting • A large file F • Linux ISO (650MB) • H(F) is available • signed by Publisher (RedHat) • A handful of untrusted sources S1,…S8 • Their aggregate BW is limited • A slew of receivers R1,...,R1,000,000 • Version 81.3 just released! Want it Now!

  6. Three Desirable Properties Clients Get Fast Downloads Sources Can Multicast Clients Can Verify Blocks On-the-Fly

  7. Receivers Get Fast, Verifiable Downloads • The trusted publisher (RedHat) • Splits up F into n blocks • Hashes all blocks • Signs all hashes (or hash tree) • Receivers: • Download and verify hashes • Download needed file blocks in parallel

  8. Everyone for Themselves S3 S2 S4 S1 R7 R9 R2 R12 R3 R8 R10 R1 R13 R4 R11 R6 R5

  9. Everyone For Themselves Clients Get Fast Downloads Sources Can Multicast Clients Can Verify Blocks On-the-Fly

  10. Verifiable Multicast (BitTorrent) S3 S2 S4 S1 R12 R7 R10 R13 R5 R6 R9 R8 R3 R1 R2 R11 R4

  11. Verifiable Multicast (BitTorrent) Clients Get Fast Downloads Sources Can Multicast Clients Can Verify Blocks On-the-Fly

  12. … ? ? ? ? ? ? ? ? ? ? … ? ? ? ? ? ? ? ? ? ? … ? ? ? ? ? ? ? ? ? ? … ? ? ? ? ? ? ? ? ? ? … ? ? ? ? ? ? ? ? ? ? Multicast With Erasure Codes • Sources erasure encode the file F blocks F n blocks

  13. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Multicast With Erasure Codes … … • Sources erasure encode the file F • Receivers collect blocks and decode … … … n blocks F blocks F n blocks 1.03n blocks

  14. Multicast With Erasure Codes S2 S3 S1 S4 R8 R9 R10 R6 R5 R12 R4 R11 R1 R3 R13 R7 R2

  15. Multicast With Erasure Codes • Bullet [SOSP 2003] • SplitStream [SOSP 2003] • Big Downloads [IPTPS 2003] • Informed Content Delivery [SIGCOMM 2002]

  16. Receivers Cannot Verify Content S2 S3 S1 S4 ? ? ? ? ? ? ? ? ? ? ? ? ? ? R1

  17. Receivers Cannot Verify Content S2 S3 S1 S4 ? ? ? ? ? ? ? ? ? ? R1

  18. Multicast With Erasure Codes Clients Get Fast Downloads Sources Can Multicast Clients Can Verify Blocks On-the-Fly

  19. Multicast With Erasure Codes Clients Get Fast Downloads Sources Can Multicast Clients Can Verify Blocks On-the-Fly

  20. What is the Attack Goal? S2 S3 • To corrupt the file. • To waste bandwidth. S1 R

  21. How To Attack? • Send correct blocks but with skewed distributions. • “Distribution Attack” • Send incorrect blocks • “Pollution Attack” • Karlof et al. [NDSS ’04] S2 S3 S1 R

  22. Properties of a Solution to Pollution S2 S3 • OK: Receivers can tell good from bad. • Much better: Receivers can finger bad blocks as they arrive. S1 R CONTRIBUTION

  23. Outline • Introduction • Review of LT Codes • Strawman #1 • Strawman #2 • Efficiently Catching Bad Blocks as They Arrive

  24. LT-Codes [Luby, FOCS 2002] b1 F= b2 b3 b4 b5 n=5input blocks

  25. LT-Codes – How To Encode • Pick degreed1 from a pre-specified distribution. (d1=2) • Select d1 input blocks uniformly at random. (Pick b1 and b4 ) • Compute their sum. • Output E(F)= c1 b1 F= b2 b3 b4 b5

  26. LT-Codes – How To Encode (cont’d) E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  27. How To Decode E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  28. How To Decode E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  29. How To Decode E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  30. How To Decode E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  31. How To Decode E(F)= c1 c2 b5 c3 b5 c4 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  32. c4 b5 How To Decode E(F)= c1 c2 b5 c3 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  33. c4 b5 How To Decode E(F)= c1 c2 b5 c3 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  34. c4 b5 How To Decode E(F)= c1 c2 b5 c3 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  35. b5 c4 How To Decode E(F)= c1 c2 b2 c3 b5 b2 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  36. b5 c4 How To Decode E(F)= c1 c2 b2 c3 b5 b2 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  37. Outline • Introduction • Review of LT Codes • Strawman #1 • Simple Solution To Tell Good Blocks From Bad • Strawman #2 • Efficiently Catching Bad Blocks as They Arrive

  38. “Smart Decoder” for LT-Codes E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  39. “Smart Decoder” for LT-Codes E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  40. “Smart Decoder” for LT-Codes E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  41. “Smart Decoder” for LT-Codes E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  42. “Smart Decoder” for LT-Codes E(F)= c1 c2 c3 c4 c5 c6 c7 b1 F= b2 b3 b4 b5

  43. “Smart Decoder” for LT-Codes E(F)= c1 c2 b5 c3 b5 c4 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  44. “Smart Decoder” for LT-Codes E(F)= c1 c2 b5 c3 b5 c4 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  45. “Smart Decoder” for LT-Codes E(F)= c1 c2 b5 c3 b5 c4 c5 b5 c6 c7 X b1 F= b2 b3 b4 b5

  46. “Smart Decoder” for LT-Codes X E(F)= c1 c2 b5 c3 c4 b5 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  47. “Smart Decoder” for LT-Codes E(F)= c1 c2 b5 c3 b5 c4 b5 c5 c6 c7 b1 F= b2 b3 b4 b5

  48. “Smart Decoder:” Problem • Data collected from 50 random Online encodings of a 10,000 block file.

  49. Outline • Introduction • Review of LT Codes • Strawman #1 • Strawman #2 • Hashing/Signing Encoded Blocks • Efficiently Catching the Bad as They Arrive

  50. ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Hashing/Signing Encoded Blocks n blocks e·n blocks F • Trusted Publisher (RedHat) • Picks e, computes e·n encoded blocks • Hashes all encoded blocks • Signs the hashes.

More Related