1 / 59

Prophecy : Using History for High-Throughput Fault Tolerance

Prophecy : Using History for High-Throughput Fault Tolerance. Siddhartha Sen Joint work with Wyatt Lloyd and Mike Freedman Princeton University. Non-crash failures happen. Model as Byzantine (malicious). Mask Byzantine faults. Service. Clients. Mask Byzantine faults. Throughput. Clients.

mika
Download Presentation

Prophecy : Using History for High-Throughput Fault Tolerance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prophecy: Using History for High-Throughput Fault Tolerance Siddhartha Sen Joint work with Wyatt Lloyd and Mike Freedman Princeton University

  2. Non-crash failures happen Model as Byzantine (malicious)

  3. Mask Byzantine faults Service Clients

  4. Mask Byzantine faults Throughput Clients Replicated service

  5. Mask Byzantine faults Throughput Linearizability (strong consistency) Clients Replicated service

  6. Byzantine fault tolerance (BFT) • Low throughput • Modifies clients • Long-lived sessions

  7. Prophecy • High throughput + good consistency • No free lunch: • Read-mostly workloads • Slightly weakened consistency

  8. Byzantine fault tolerance (BFT) • Low throughput • Modifies clients • Long-lived sessions D-Prophecy Prophecy

  9. Traditional BFT reads application Agree? … Clients Replica Group

  10. A cache solution cache application Agree? … Clients Replica Group

  11. A cache solution cache application • Problems: • Huge cache • Invalidation Agree? … Clients Replica Group

  12. A compact cache cache application … … … Clients Replica Group

  13. A compact cache cache application … … … Clients Replica Group

  14. A sketcher sketcher application … Clients Replica Group

  15. A sketcher sketch webpage ………… ………… … Clients ………… Replica Group

  16. Executing a read sketch webpage ………… Agree? • Fast, load-balanced reads ………… ………… … Clients ………… Replica Group

  17. Executing a read sketch webpage ………… Agree? ………… ………… … Clients ………… Replica Group

  18. Executing a read sketch webpage key-value store ………… replicated state machine ………… … Clients ………… Replica Group

  19. Executing a read sketch webpage ………… Agree? Maintain a fresh cache ………… ………… … Clients ………… Replica Group

  20. NO! Did we achieve linearizability?

  21. Executing a read sketch webpage ………… ………… ………… … Clients ………… Replica Group

  22. Executing a read sketch webpage ………… Agree? ………… ………… … Clients ………… Replica Group

  23. Executing a read sketch webpage ………… Agree? Fast reads may be stale ………… ………… … Clients ………… Replica Group

  24. Load balancing sketch webpage ………… ………… Agree? Pr(k stale) = gk ………… … Clients ………… Replica Group

  25. D-Prophecy vs. BFT Traditional BFT: • Each replica executes read • Linearizability D-Prophecy: • One replica executes read • “Delay-once” linearizability … Clients Replica Group

  26. Byzantine fault tolerance (BFT) • Low throughput • Modifies clients • Long-lived sessions D-Prophecy Prophecy

  27. Key-exchange overhead 11% 3%

  28. Internet services … Clients Replica Group

  29. A proxy solution Sketcher Proxy Consolidate sketchers … Clients Replica Group

  30. A proxy solution Sketcher Sketcher must be fail-stop … Clients Trusted Replica Group

  31. A proxy solution Sketcher Sketcher must be fail-stop • Trust middlebox already • Small and simple … Clients Trusted Replica Group

  32. Prophecy Executing a read Sketcher Fast, load-balanced reads ………… ………… ………… ………… q ………… … Clients Trusted ………… Replica Group

  33. Prophecy Sketcher Fast reads may be stale ………… ………… ………… ………… ………… ………… … Clients Trusted ………… ………… Replica Group

  34. Delay-once linearizability

  35. Delay-once linearizability Read-after-write property  W, R, W, W, R, R, W, R

  36. Delay-once linearizability Read-after-write property  W, R, W, W, R, R, W, R

  37. Example application • Upload embarrassing photos 1. Remove colleagues from ACL 2. Upload photos 3. (Refresh) • Weak may reorder • Delay-once preserves order

  38. Byzantine fault tolerance (BFT) • Low throughput • Modifies clients • Long-lived sessions D-Prophecy Prophecy

  39. Implementation • Modified PBFT • PBFT is stable, complete • Competitive with Zyzzyva et. al. • C++, Tamer async I/O • Sketcher: 2000 LOC • PBFT library: 1140 LOC • PBFT client: 1000 LOC

  40. Evaluation • Prophecy vs. proxied-PBFT • Proxied systems • D-Prophecy vs. PBFT • Non-proxied systems

  41. Evaluation • Prophecy vs. proxied-PBFT • Proxied systems • We will study: • Performance on “null” workloads • Performance with real replicated service • Where system bottlenecks, how to scale

  42. Basic setup Sketcher (concurrent) Clients (100) Replica Group (PBFT)

  43. Alexa top sites: < 15% Fraction of failed fast reads

  44. Small benefit on null reads

  45. Apache webserver setup Sketcher Clients Replica Group

  46. Large benefit on real workload 3.7x 2.0x

  47. Benefit grows with work 94s (Apache) Null workloads are misleading!

  48. Benefit grows with work

  49. Single sketcher bottlenecks

  50. Scaling out

More Related