1 / 55

Virtual Queues as a Trade Processing Pattern

Virtual Queues as a Trade Processing Pattern. Uri Cohen @uri1803 | github.com /uric Head of Product @ GigaSpaces. Event Processing at Massive Scale Approaches to Concurrency . Uri Cohen @uri1803 | github.com /uric Head of Product @ GigaSpaces. This is What It Used to Be Like.

jabari
Download Presentation

Virtual Queues as a Trade Processing Pattern

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Virtual Queues as a Trade Processing Pattern Uri Cohen @uri1803 | github.com/uric Head of Product @ GigaSpaces

  2. Event Processing at Massive Scale Approaches to Concurrency Uri Cohen @uri1803 | github.com/uric Head of Product @ GigaSpaces

  3. This is What It Used to Be Like

  4. That’s What It’s Like Now

  5. 15 Billion Trades / Day on NYSE alone Some Numbers http://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html

  6. That’s 641K Trades / Second Some Numbers http://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html

  7. 12 Billion Shares change hands every day Some Numbers http://www.bloomberg.com/news/2012-01-23/stock-trading-is-lowest-in-u-s-since-2008.html

  8. $4 Million The cost of 1 millisecond of latency to a broker Some Numbers http://www.tabbgroup.com/PublicationDetail.aspx?PublicationID=346

  9. Massive stream of events Time is money, literally Can’t lose a single message Fairness is a must The Problem

  10. Order Book -Simplistic Example

  11. Price: $10 Order Book -Simplistic Example

  12. Order Book -Simplistic Example Price: $10

  13. Order Book -Simplistic Example Price: $10

  14. Order Book -Simplistic Example Price: $10

  15. Price: $11 Order Book -Simplistic Example

  16. Order Book -Simplistic Example Price: $11

  17. Low latencyIn memory, GC tuning Scalability Multi-core Multi-node Ordering By price, order time Exclusivity Resiliency What it Really Means

  18. All things FCFS, with a limited stock Flight booking Betting Online Auctions Cloud Spot Instances eCommerce Trading is Just One Use Case

  19. Let’s Talk Solutions

  20. Queue (SEDA/Actor Style) Not Validated Validated Processed Validator Processor

  21. The Good: Ordered (Is it fair?) Multi-threaded The Bad: Not very scalable Locking Context switching Transient Queue (SEDA /Actor Style)

  22. The Cost of Locking http://disruptor.googlecode.com/files/Disruptor-1.0.pdf

  23. Queue (Lack of) Fairness 50 60 Consumer Thread 1 Consumer Thread 2

  24. Queue (Lack of) Fairness 50 60 Consumer Thread 1 Consumer Thread 2

  25. Queue (Lack of) Fairness Can you tell which order will be executed 1st? 50 60 Consumer Thread 1 Consumer Thread 2

  26. Single-Threaded Queue Validator Processor

  27. The Good: Fast, very fastNo contention No context switches Always fair The Bad: Multi-core? Not fit for Intense compute & I/ONeed to be async. Transient Single- Threaded Queue

  28. They do it… Single- Threaded Queue

  29. Disruptor (LMAX)

  30. Segmented Queue Processor thread pool per segment Symbol=A-H Symbol=I-S Symbol=T-Z Validator Processor Processor

  31. Segmented Queue - Optimization Single Processor thread pool, pick random segment Symbol=A-H Symbol=I-S Symbol=T-Z Processor

  32. The Good: ScalableBut segments can get hot Minimizes contention The Bad: Not trivial to implement Still unfairIs total ordering needed? Transient Segmented Queue

  33. Exclusivity is Key Process one message for each segment at the same time No exclusivity across segments What about Fairness?

  34. Implicit Exclusivity Single processor thread per segment Symbol=A-H Symbol=I-S Symbol=T-Z Processor Processor Processor

  35. Explicit Exclusivity Shared thread pool, mark segments under processing (CAS) Segment 1 Segment 2 Segment 3 Processor

  36. Explicit Exclusivity Shared thread pool, mark segments under processing (CAS) Segment 1 Segment 2 Segment 3 Processor

  37. Explicit Exclusivity Shared thread pool, mark segments under processing (CAS) Segment 1 Segment 2 Segment 3 Processor

  38. Explicit Exclusivity Shared thread pool, mark segments under processing (CAS) Segment 1 Segment 2 Segment 3 Processor

  39. Num. of segments is keyToo few: little concurrency Too many: wasting memory Explicit Exclusivity

  40. Dynamic Segmentation Segments are created and removed as needed Processor

  41. Dynamic Segmentation Segments are created and removed as needed “GOOG” Processor

  42. Dynamic Segmentation Segments are created and removed as needed “GOOG” Processor

  43. Dynamic Segmentation Segments are created and removed as needed “GOOG” “AAPL” Processor

  44. Dynamic Segmentation Segments are created and removed as needed “AMZN” “GOOG” “AAPL” Processor

  45. Segments created as needed Randomize on segments until available one found Fast, scalable, fair We call it “FIFO groups” or “Virtual Queues” Dynamic Segmentation

  46. Memory state can get corrupt on errors It’s not always as simple as “pop off the queue” limits, priorities, circuit breakers, etc. Resiliency is always a pain It Can (and Does) Get Much More Complex

  47. What you don’t want to do Implement data structures Handle concurrency Handle HA Handle transactions A Bit about Usability

  48. What you want to control Event flow Grouping attribute (e.g symbol) Event handlers A Bit about Usability

  49. Transactional Highly available Supports complex matching Data Grid as a Foundation

  50. How We Thought of It

More Related