1 / 69

Disorderly programming for a distributed world

Disorderly programming for a distributed world. Peter Alvaro UC Berkeley. Joint work . Peter Alvaro Neil Conway Joseph M. Hellerstein William R. Marczak National Science Foundation Air Force Office of Scientific Research Gifts from IBM, Microsoft and Yahoo! Research. Thanks to.

lola
Download Presentation

Disorderly programming for a distributed world

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Disorderly programming for a distributed world Peter Alvaro UC Berkeley

  2. Joint work • Peter Alvaro • Neil Conway • Joseph M. Hellerstein • William R. Marczak • National Science Foundation • Air Force Office of Scientific Research • Gifts from IBM, Microsoft and Yahoo! Research Thanks to

  3. The future is already here • All systems are (or are becoming) distributed • Programming distributed systems is hard • Reasoning about them is harder

  4. Outline • Disorderly Programming • The Bloom programming language • CALM Analysis and visualization • Challenge app: replicated shopping carts

  5. Programming distributed systems

  6. The state of the art Order is pervasive in the Von Neumann model • Program state is an ordered array • Program logic is a sequence of instructions

  7. The state of the art Order is pervasive in the Von Neumann model Parallelism and concurrency via retrofits: • Threads • Event-driven programming

  8. The state of the art In distributed systems, order is

  9. The state of the art In distributed systems, order is • expensive to enforce

  10. The state of the art In distributed systems, order is • expensive to enforce • often unnecessary

  11. The state of the art In distributed systems, order is • expensive to enforce • often unnecessary • easy to get wrong

  12. The art of the state The state of the art

  13. The art of the state Disorderly programming

  14. The art of the state Disorderly programming: Computation as transformation

  15. The art of the state Disorderly programming: • Program state is unordered collections • Program logic is an unordered ``bag’’ of rules

  16. The art of the state Disorderly programming: • Independence and concurrency are assumed • Ordering is explicit

  17. The art of the state Disorderly programming

  18. BUD: Bloom Under Development • Ruby internal DSL • Set comprehension style of programming • Declarative semantics

  19. Operational model

  20. Bloom Rules multicast (message * members) do|mes, mem| [mem.address, mes.id, mes.payload] end <~

  21. Bloom Rules multicast <~ (message * members) do|mes, mem| [mem.address, mes.id, mes.payload] end

  22. Bud language features • Module system • Encapsulation and composition via mixins • Abstract interfaces, concrete implementations • Metaprogramming and reflection • The program is data • Pay-as-you-code schemas • Default is key => value • CALM Analysis

  23. Writing distributed programs in Bloom

  24. Abstract Interfaces and Declarations moduleDeliveryProtocol state do interface input, :pipe_in, [:dst, :src, :ident]=>[:payload] interface output, :pipe_sent, pipe_in.schema interface output, :pipe_out, pipe_in.schema end end

  25. Concrete Implementations moduleBestEffortDelivery includeDeliveryProtocol state do channel :pipe_chan, pipe_in.schema end bloom :snd do pipe_chan<~pipe_in end bloom :done do pipe_sent<=pipe_in pipe_out<= pipe_chan end end Asynchrony

  26. A simple key/value store moduleKVSProtocol state do interface input, :kvput, [:key]=>[:reqid, :value] interface input, :kvdel, [:key]=>[:reqid] interface input, :kvget, [:reqid]=>[:key] interface output, :kvget_response, [:reqid]=>[:key, :value] end end

  27. A simple key/value store moduleBasicKVS includeKVSProtocol state do table :kvstate, [:key]=>[:value] end bloom :mutatedo kvstate<+kvput{|s|[s.key, s.value]} kvstate<- (kvstate*kvput).lefts(:key=>:key) end bloom :getdo temp :getj<= (kvget*kvstate).pairs(:key=>:key) kvget_response<=getjdo|g, t| [g.reqid, t.key, t.value] end end bloom:deletedo kvstate<- (kvstate*kvdel).lefts(:key=>:key) end end Nonmonotonic operation

  28. CALM Analysis

  29. Asynchronous messaging You never really know

  30. Asynchronous messaging A B A B send C C

  31. Monotonic Logic The more you know, the more you know.

  32. Monotonic Logic A B A select/ filter C C D E E

  33. Monotonic Logic A B f(A) f(B) project / map C f(C)

  34. Monotonic Logic A B C D E B join / compose D B F D

  35. Monotonic Logic is order-insensitive A B A C C D E E

  36. Monotonic Logic is pipelineable A B A C C D E E

  37. Nonmonotonic Logic When do you know for sure?

  38. NonmonotonicLogic Retraction! A B X C D E B A set minus Z C E D B Y D Retraction!

  39. Nonmonotonic logic is order-sensitive A B C D E A set minus C E B D

  40. Nonmonotonic logic is blocking A A ? set minus A?

  41. Nonmonotonic logic is blocking A set minus A ? ``Sealed’’

  42. CALM Analysis • Asynchrony => loss of order • Nonmonotonicity => order-sensitivity • Asynchrony ; Nonmonotonicity => Inconsistency […]

  43. CALM Analysis • Asynchrony => loss of order • Nonmonotonicity => order-sensitivity • Asynchrony ; Nonmonotonicity => Inconsistency ``Point of Order’’ […] ?

  44. Resolving points of order

  45. Resolving points of order • Ask for permission

  46. Resolving points of order • Ask for permission Coordination => strong consistency

  47. Resolving points of order • Ask for permission • Ask for forgiveness

  48. Resolving points of order • Ask for permission • Ask for forgiveness Compensation, weak consistency

  49. Resolving points of order • Ask for permission • Ask for forgiveness • Ask differently? Rewrite to reduce consistency cost…

  50. Shopping Carts

More Related