1 / 43

Phase Reconciliation for Contended In-Memory Transactions

Phase Reconciliation for Contended In-Memory Transactions. Neha Narula, Cody Cutler, Eddie Kohler, Robert Morris MIT CSAIL and Harvard. Cloud Computing and Databases. Two trends: multicore and in-memory databases

shana-logan
Download Presentation

Phase Reconciliation for Contended In-Memory Transactions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Phase Reconciliation for Contended In-Memory Transactions Neha Narula, Cody Cutler, Eddie Kohler, Robert Morris MIT CSAIL and Harvard

  2. Cloud Computing and Databases Two trends: multicore and in-memory databases Multicore databases face increased contention due to skewed workloads and rising core counts

  3. BEGIN Transaction ADD(x, 1) ADD(y, 2) ENDTransaction

  4. Throughput on a Contentious Transactional Workload

  5. Throughput on a Contentious Transactional Workload

  6. Concurrency Control Forces Serialization core 0 ADD(x)ADD(y) core 1 ADD(x)ADD(y) core 2 ADD(x)ADD(y) time

  7. BEGIN Transaction ADD(x, 1) ADD(y, 2) ENDTransaction

  8. Multicore OS Scalable Counters Kernel can apply increments in parallel using per-core counters x0 = x0+1; core 0 core 1 x1 = x1+1; core 2 x2 = x2+1; time

  9. Multicore OS Scalable Counters To read per-core data, system has to stop all writes and reconcile x x0 = x0+1; core 0 core 1 x1 = x1+1; core 2 print(x); x2 = x2+1; x=x0+x1+x2; print(x) time

  10. Can we use per-core data in complex database transactions?

  11. BEGIN Transaction ADD(x, 1) ADD(y, 2) ENDTransaction BEGIN Transaction GET(x) GET(y) ENDTransaction

  12. Challenges • Deciding what records should be split among cores • Transactions operating on some contentious and some normal data • Different kinds of operations on the same records

  13. Phase Reconciliation • Database automatically detects contention to split a record among cores • Database cycles through phases: split and joined • OCC serializes access to non-split data • Split records have assigned operations for split phases Doppel, an in-memory transactional database

  14. Outline • Phases • Splittable operations • Doppel optimizations • Performance evaluation

  15. Split Phase split phase • A transaction can operate on split and non-split data • Split records are written to per-core (x) • Rest of the records use OCC (u, v, w) • OCC ensures serializability for the non-split parts of the transaction core 0 ADD(x) ADD(u) GET(x)GET(v) x0=x0+1 core 1 ADD(x) ADD(v) x1=x1+1 core 2 ADD(x) ADD(w) x2=x2+1

  16. Reconciliation reconciliation split phase • Cannot correctly process a read of x in current state • Abort read transaction • Reconcile per-core data to global store x=x+x0 x0=0 core 0 ADD(x) ADD(u) GET(x)GET(v) x0=x0+1 core 1 ADD(x) ADD(v) x=x+x1 x1=0 x1=x1+1 core 2 ADD(x) ADD(w) x=x+x2 x2=0 x2=x2+1

  17. Joined Phase joined phase reconciliation split phase • Wait until all processes have finished reconciliation • Resume aborted read transactions using OCC • Process all new transactions using OCC • No split data x=x+x0 x0=0 core 0 GET(x) GET(v) ADD(x) ADD(u) GET(x)GET(v) x0=x0+1 core 1 ADD(x) ADD(v) x=x+x1 x1=0 x1=x1+1 core 2 ADD(x) ADD(w) x=x+x2 x2=0 x2=x2+1

  18. Resume Split Phase joined phase reconciliation split phase split phase • When done, transition to split phase again • Wait for all processes to acknowledge they have completed joined phase before writing to per-core data x=x+x0 x0=0 core 0 GET(x) GET(v) ADD(x) ADD(u) GET(x)GET(v) x0=x0+1 core 1 ADD(x) ADD(v) ADD(x) ADD(v) x=x+x1 x1=0 x1=x1+1 x1=x1+1 ADD(x) ADD(v) core 2 ADD(x) ADD(w) x=x+x2 x2=0 x2=x2+1 x2=x2+1

  19. Outline • Phases • Splittable operations • Doppel Optimizations • Performance evaluation

  20. Transactions and Operations BEGIN Transaction (y) ADD(x,1) ADD(y,2) ENDTransaction • Transactions are composed of one or more operations • Only some operations are amenable to splitting

  21. Supported Operations Splittable ADD(k,n) MAX(k,n) MULT(k,n) OPUT(k,v,o) TOPKINSERT(k,v,o) Not Splittable GET(k) PUT(k,v)

  22. MAX Example • Each core keeps one piece of summarized state xi • MAX is commutative so results can be determined out of order MAX(x,55) core 0 x0 = 55 x0 = MAX(x0,55) MAX(x,10) core 1 x1 = 10 x1 = MAX(x1,10) core 2 MAX(x,21) x2 = 21 x2 = MAX(x2,21)

  23. MAX Example • Each core keeps one piece of summarized state xi • MAX is commutative so results can be determined out of order MAX(x,55) MAX(x,2) core 0 x0 = 55 x0 = MAX(x0,55) x1 = MAX(x1,2) MAX(x,10) MAX(x,27) core 1 x1 = 27 x1 = MAX(x1,10) x1 = MAX(x1,27) core 2 MAX(x,21) x2 = 21 x2 = MAX(x2,21) x = 55

  24. MAX Example • Each core keeps one piece of summarized state xi • MAX is commutative so results can be determined out of order MAX(x,55) MAX(x,2) core 0 x0 = 55 x0 = MAX(x0,55) x1 = MAX(x1,2) MAX(x,27) MAX(x,10) core 1 x1 = 27 x1 = MAX(x1,27) x1 = MAX(x1,10) MAX(x,21) core 2 x2 = 21 x2 = MAX(x2,21) x = 55

  25. What Can Be Split? Operations that can be split must be: • Commutative • Pre-reconcilable • On a single key

  26. What Can’t Be Split? • Operations that return a value • ADD_AND_GET(x,4) • Operations on multiple keys • ADD(x,y) • Different operations in the same phase, even if arguments make them commutative • ADD(x,0) and MULT(x,1)

  27. Outline • Phases • Splittable operations • Doppel Optimizations • Performance evaluation

  28. Batching Transactions joined phase reconciliation split phase x=x+x0 x0=0 core 0 GET(x) GET(v) ADD(x) ADD(u) GET(x)GET(v) x0=x0+1 core 1 ADD(x) ADD(v) x=x+x1 x1=0 x1=x1+1 core 2 ADD(x) ADD(w) x=x+x2 x2=0 x2=x2+1

  29. Batching Transactions joined phase reconciliation split phase • Don’t switch phases immediately; stash reads • Wait to accumulate stashed transactions • Batch for joined phase x=x+x0 x0=0 core 0 GET(x) GET(v) ADD(x) ADD(u) GET(x)GET(v) x0=x0+1 core 1 ADD(x) ADD(v) ADD(x) ADD(v) x=x+x1 x1=0 x1=x1+1 x1=x1+1 GET(x) GET(x) core 2 GET(x) ADD(x) ADD(w) GET(x) x=x+x2 x2=0 x2=x2+1

  30. Ideal World • Transactions with contentious operations happen in the split phase • Transactions with incompatible operations happen correctly in the joined phase

  31. How To Decide What Should Be Split Data • Database starts out with no split data • Count conflicts on records • Make key split if #conflicts > conflictThreshold • Count stashes on records in the split phase • Move key back to non-split if #stashes too high

  32. Outline • Phases • Splittable operations • Data classification • Performance evaluation

  33. Performance • Contention vs. parallelism • What kinds of workloads benefit? • A realistic application: RUBiS Doppel implementation: • Multithreaded Go server; worker thread per core • All experiments run on an 80 core intel server running 64 bit Linux 3.12 • Transactions are one-shot procedures written in Go

  34. Parallel Performance on Conflicting Workloads Throughput (millions txns/sec) 20 cores, 1M 16 byte keys, transaction: ADD(x,1)

  35. Increasing Performance with More Cores 1M 16 byte keys, transaction: ADD(x,1) all writing same key

  36. Varying Number of Hot Keys 20 cores, transaction: ADD(x,1)

  37. How Much Stashing Is Too Much? 20 cores, transactions: LIKE read, LIKE write

  38. RUBiS • Auction application modeled after eBay • Users bid on auctions, comment, list new items, search • 1M users and 33K auctions • 7 tables, 26 interactions • 85% read only transactions • More contentious workload • Skewed distribution of bids • More writes

  39. StoreBid Transaction BEGIN Transaction (bidder, amount, item) num_bids_item = num_bids_item + 1 if amount > max_bid_item: max_bid_item= amount max_bidder_item= bidder bidder_item_ts= Bid{bidder, amount, item, ts} ENDTransaction

  40. StoreBid Transaction BEGIN Transaction (bidder, amount, item) ADD(num_bids_item,1) MAX(max_bid_item, amount) OPUT(max_bidder_item, bidder, amount) PUT(new_bid_key(), Bid{bidder, amount, item, ts}) ENDTransaction All commutative operations on potentially conflicting auction metadata

  41. RUBiS Throughput Throughput (millions txns/sec) 20 cores, 1M users 33K auctions

  42. Related Work • Split counters in multicore OSes • Linux kernel • Commutativity in distributed systems and concurrency control • [Shapiro ‘11] • [Li ‘12] • [Lloyd ‘12] • [Weihl‘88] • Optimistic concurrency control • [Kung ’81] • [Tu ‘13]

  43. Summary • Contention is rising with increased core counts • Commutative operations are amenable to being applied in parallel, per-core • We can get good parallel performance on contentious transactional workloads by combining per-core split data with concurrency control Neha Narula http://nehanaru.la @neha

More Related