1 / 27

Molecular Transactions

Molecular Transactions. G. Ramalingam Kapil Vaswani Rigorous Software Engineering, MSRI. Simple banking application. ACID Transaction. let transfer from to amount = atomic { match hasBalance from amount with | true -> debit from amount credit to amount SUCCESS

brilliant
Download Presentation

Molecular Transactions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Molecular Transactions G. Ramalingam Kapil Vaswani Rigorous Software Engineering, MSRI

  2. Simple banking application ACID Transaction let transfer from to amount = atomic { matchhasBalance from amount with | true -> debit from amount credit to amount SUCCESS | false -> INSUFFICIENT_BALANCE } Database Application server Rigorous Software Engineering Microsoft Research, India

  3. ? Rigorous Software Engineering Microsoft Research, India

  4. Scalable applications • Support lots of users • TBs of data! • Many concurrent transactions • Unpredictable workload • Key requirements • Available • Fault tolerant • Data consistency • Achieving all at the same time non-trivial! Application server Relational Database Rigorous Software Engineering Microsoft Research, India

  5. Designing for scale Rigorous Software Engineering Microsoft Research, India

  6. Scaling application tier Application server Database Rigorous Software Engineering Microsoft Research, India

  7. Replication Application server Replica Replica Replica Rigorous Software Engineering Microsoft Research, India

  8. Data partitioning Application server A first-class citizen in cloud based storage systems Data partition Data partition Rigorous Software Engineering Microsoft Research, India

  9. Horizontal data partitioning let transfer from to amount = let status = atomic { matchhasBalance from amount with | true -> debit from amount DEBIT_COMPLETE | false -> INSUFFICIENT_BALANCE } Distributed vs. local transactions Database partition Distributed transactions do not scale  Azure tables, BigTable, Dynamo do not support distributed transactions! atomic{ matchstatus with | DEBIT_COMPLETE -> credit to amount SUCESSS | _ -> INSUFFICIENT_BALANCE } Database partition Rigorous Software Engineering Microsoft Research, India

  10. Consistency let transfer from to amount = let status = atomic { matchhasBalance from amount with | true -> debit from amount DEBIT_COMPLETE | false -> INSUFFICIENT_BALANCE } Database partition atomic { match status with | DEBIT_COMPLETE -> credit to amount SUCESSS | _ -> INSUFFICIENT_BALANCE } Database partition Rigorous Software Engineering Microsoft Research, India

  11. Rigorous Software Engineering Microsoft Research, India

  12. Compute failures let transfer from to amount = let status = atomic { matchhasBalance from amount with | true -> debit from amount DEBIT_COMPLETE | false -> INSUFFICIENT_BALANCE } Database partition Server failures Database partition Rigorous Software Engineering Microsoft Research, India

  13. Handling compute failures let transfer from to amount = let status = atomic { matchhasBalance from amount with | true -> debit from amount DEBIT_COMPLETE | false -> INSUFFICIENT_BALANCE } Database partition Checkpoint storage atomic{ matchstatus with | DEBIT_COMPLETE -> credit to amount SUCESSS | _ -> INSUFFICIENT_BALANCE } Database partition Rigorous Software Engineering Microsoft Research, India

  14. There’s more…logical failures let transfer from to amount = let status = atomic { matchhasBalance from amount with | true -> debit from amount DEBIT_COMPLETE | false -> INSUFFICIENT_BALANCE } Database partition What if credit fails? e.g. account closed Database partition Rigorous Software Engineering Microsoft Research, India

  15. Handling logical failures withcompensating actions let transfer from to amount = let status = atomic { matchhasBalance from amount with | true -> debit from amount DEBIT_COMPLETE | false -> INSUFFICIENT_BALANCE } let compensate from amount = atomic { credit from amount } Credit fails Rigorous Software Engineering Microsoft Research, India

  16. Molecular transactions Fault-tolerant composition of atomic transactions with compensating actions molecule { atomic { } atomic { } ⇋ ⊗ atomic { } atomic { } ⇋ } Rigorous Software Engineering Microsoft Research, India

  17. Account transfer with MT let transfer from to amount = molecule{ do! atomic { match hasBalancefrom amount with | true -> debit from amount | false -> abort } |> compensateWith<| atomic { credit from amount } return! atomic { match exists to with | true -> credit to amount | false -> abort } } • Hides • Serialization • Messaging • Fault tolerance First atomic step Compensation • No isolation • Compensating actions must semantically undo Second atomic step Rigorous Software Engineering Microsoft Research, India

  18. Implementation Rigorous Software Engineering Microsoft Research, India

  19. System model • Compute agents • May fail at any time • System detects failure and restarts agents • All agent-local state is lost • Partitioned storage • Partitions are units of serializability • Persistent queue • Communication between agents Rigorous Software Engineering Microsoft Research, India

  20. Persistent queue • Not really a queue! • Out-of-order • At-least once • Operations • Enqueue(m) • Get() • Gets a message, makes message invisible • Message re-appears if not dequeued • Dequeue(m) • Permenantly deletes the message Rigorous Software Engineering Microsoft Research, India

  21. Tolerating agent failures • Agent can fail between transactions • Check-pointing • Check-point state after every transaction Rigorous Software Engineering Microsoft Research, India

  22. Check-pointing Enqueue Read Dequeue Database partition Database partition Rigorous Software Engineering Microsoft Research, India

  23. Compute failures Enqueue Read • Transactions may be evaluated more than once! • Messages may be sent more than once! Dequeue Database partition Rigorous Software Engineering Microsoft Research, India

  24. Idempotence • Instrument transactions for idempotence • Assumes key-value store with a unique key constraint • Assign each atomic step in each MT a unique id • Add (unique id, return value) to write set • Idempotence log in transaction’s partition • If transaction evaluates more than once • Unique key violation causes transaction failure • Return stored value Rigorous Software Engineering Microsoft Research, India

  25. Summary • Molecular transactions • Language abstraction for programming weakly consistent applications • Hides messaging, queuing, fault tolerance • Expressed using a monad, lends to declarative, bug-free code • Efficient implementation on cloud based storage systems • Does not require 2PC! • Built a number of applications on Azure • < 10% overheads over hand-written code Rigorous Software Engineering Microsoft Research, India

  26. Open questions • Specifying and reasoning about correctness of molecular transactions • Declarative ways of making consistency/performance trade-offs • Do cloud-based systems provide right primitives? Rigorous Software Engineering Microsoft Research, India

  27. Thank you Questions? Rigorous Software Engineering Microsoft Research, India

More Related