1 / 23

Optimizing RAM-latency Dominated Applications

Optimizing RAM-latency Dominated Applications. Yandong Mao , Cody Cutler, Robert Morris MIT CSAIL. RAM-latency may dominate performance. RAM-latency dominated applications follow long pointer chains working set >> on-chip cache A lot of cache misses -> stalling on RAM fetches

jnale
Download Presentation

Optimizing RAM-latency Dominated Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Optimizing RAM-latency Dominated Applications Yandong Mao, Cody Cutler, Robert Morris MIT CSAIL

  2. RAM-latency may dominate performance • RAM-latency dominated applications • follow long pointer chains • working set >> on-chip cache • A lot of cache misses -> stalling on RAM fetches • Example: Garbage Collector • Identify live objects by following inter-object pointers • Spend much of its time stalling to follow pointers, due to RAM latency

  3. Addressing RAM-latency bottleneck? • View RAM as we view disk • High latency • A similar set of optimization techniques • Batching • Sorting • Access I/O in parallel and asynchronously

  4. Outline • Hardware Background • Three techniques to address RAM-latency • Linearization: Garbage Collector • Interleaving: Masstree • Parallelization: Masstree • Discussion • Conclusion

  5. Three Relevant Hardware Features • Intel Xeon X5690 • 1. Fetch RAM before needed • Hardware prefetcher – sequential or strided access pattern • Software prefetch • Out-of-order execution 2 3 5 0 1 4 RAM Controller 2. Parallel accesses to different channels Channel 0 Channel 1 Channel 2 3. Row buffer cache inside memory channel

  6. Per-array row buffer cache • Each channel has many of arrays shown below • Each array has an additional row: row buffer • Memory access: check row buffer, reload if miss Data Rows Hit in row buffer: 2x-5x faster than miss! Sequential access: 3.5x higher throughput than random access! Row buffer 4096 bytes

  7. Linearization memory accesses for Garbage Collector • Garbage Collector goal • Find live objects (tracing) • starts from root (stack, global variables) • follows object pointers of live objects • Claim space for unreachable objects • Bottleneck of tracing: RAM-latency • Pointer addresses are unpredictable and non-sequential • Each access -> cache miss -> stall for RAM-fetch

  8. Observation • Arrange objects in tracing order during garbage collection • Subsequent tracing would access memory in sequential order • Take advantage of two hardware features • Hardware prefechers: prefetch into cache • Higher row buffer hit rate

  9. Benchmark and result • Time of tracing 1.8 GB of live data • HSQLDB 2.2.9: a RDBMS engine in Java • Compacting Collector of Hotspot JVM from OpenJDK7u6 • Use copy collection to reorder objects in tracing order • Result: tracing in sequential order is 1.3X faster than random order • Future work • better linearizing algorithm than copy collection algorithm (use twice the memory!) • measure application-level performance improvement

  10. Interleaving on Masstree • Not always possible to linearize memory access • Masstree: a high performance in-memory key value store for multi-core • All cores share a single B+tree • Each core: a dedicated working thread • Scales well on multi-core • Focus on Masstree with single-thread for now

  11. Single-threaded Masstree is RAM-latency dominated • Careful design to avoid RAM fetches • trie of B+trees, inline key fragments and children in tree nodes • Accessing one fat B+tree node in one RAM-latency • Still RAM-latency dominated! • Each key-lookup follows a random path • O(N) RAM-latency (hundreds of cycles) per-lookup • A million lookups per second

  12. Batch and interleave tree lookups • Batch key lookups • Interleave computation and RAM fetch using software prefetch

  13. Find child containing A in E prefetch(B) 2. Find child containing X in E prefetch(F) E 3. Find child containing A in B prefetch(A) B is already in cache! 4. Find child containing X in F prefetch(X) F is already in cache! F B X D A • Perform a batch of lookups w/o stalling on RAM-fetch! • As long as computation (inspecting a batch of nodes) > RAM-latency • 30% improvement with batch of five

  14. Parallelizing Masstree • Interesting observation • applications are limited by RAM-latency, not by CPU • but adding more cores help! • Reason • RAM is a parallel system • More cores keeps RAM busier • Compare with interleaving technique • Same effect: keep RAM busier • Difference: from one core, and from multi-cores

  15. Parallelization improves performance by issuing more RAM loads

  16. Interleaving and Parallelization can be complementary Beats Masstree by 12-30% Improvement decreases with more cores Parallelization alone can saturate

  17. Discussion • Applicability • Lessons • Interleaving seems more general than linearization • applied to Garbage Collector? • Interleaving is more difficult than parallelization • requires batching and concurrency control • Challenges in automatic interleaving • Need to identify and resolve conflicting access • Difficult or impossible without programmers’ help

  18. Discussion • Interleaving on certain data structures • Data structures and potential applications • B+tree: Masstree • other applications use in-memory B+tree? • Hashtable: Memcached • A single hashtable • Multi-get API: natural batching and interleaving • Preliminary result: interleaving hashtable improves throughput by 1.3X

  19. Discussion • Profiling tools • Linux perf • Look at most expensive function • Manually inspect • Maybe misleading • computation limited or RAM-latency limited? • RAM stalls based tool?

  20. Related Work • PALM[Jason11]: B+tree with same interleaving technique • RAM parallelization at different levels: regulation considered harmful[Park13]

  21. Conclusion • Identifies a class of applications: dominated by RAM-latency • Three techniques to address RAM-latency bottleneck of two applications • Improve your program similarly?

  22. Questions?

  23. Single-threaded Masstree is RAM-latency dominated … B+tree, indexed by k[0:7] Trie: a tree where each level is indexed by fixed-length key fragment … B+tree, indexed by k[8:15]

More Related