1 / 15

Speculative execution

Speculative execution. Landon Cox April 5, 2017. Making disk accesses tolerable. Basic idea Remove disk accesses from critical path Transform disk latencies into memory latencies Requires predicting the future Caching Assume future accesses will look like past accesses. Process.

nigel
Download Presentation

Speculative execution

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Speculative execution Landon Cox April 5, 2017

  2. Making disk accesses tolerable • Basic idea • Remove disk accesses from critical path • Transform disk latencies into memory latencies • Requires predicting the future • Caching • Assume future accesses will look like past accesses Process Operating system Get D1 Get D1 D1 D1 D1 D2 How long does the initial access take? ~10ms

  3. Making disk accesses tolerable • Basic idea • Remove disk accesses from critical path • Transform disk latencies into memory latencies • Requires predicting the future • Caching • Assume future accesses will look like past accesses Process Operating system Get D1 D1 D1 D1 D2 How long does the subsequent access take? ~10ns (~million x faster)

  4. Making disk accesses tolerable • Basic idea • Remove disk accesses from critical path • Transform disk latencies into memory latencies • Requires predicting the future • Pre-fetching • Guess future accesses based on past accesses Process Operating system Get D1 Get D1, D2 D1 D1 D2 D1 D2

  5. Making disk accesses tolerable • Basic idea • Remove disk accesses from critical path • Transform disk latencies into memory latencies • Requires predicting the future • Pre-fetching • Guess future accesses based on past accesses Process Operating system Get D2 D1 D2 D1 D2 D1 D2

  6. Hiding disk latency • Caching works when • The recent past looks like the near future • Amortize initial retrieval penalty over many other accesses • Pre-fetching makes sense when it is • Accurate: anticipate accesses before they occur • Cheap: pre-fetching must not hinder other activities • Metrics for better pre-fetching • More accurate predictions • Reasonable cost to obtain and act on those predictions

  7. Approaches to pre-fetching • Usually rely on heuristics • Doesn’t require apps to be modified • Sequential read-ahead or history-based • However, heuristics • Can be very bad for many classes of apps • E.g., databases and small files • May require extra resources to maintain history • Or let the programmer issue hints. Positives and negatives? • Programmer knows best (better accuracy) • Must rewrite app, which could involve significant restructuring • Source code may not be available

  8. Speculative execution • “Dynamic self-analysis” • Allow app to run into the future on fake data • Speculative execution reveals future requests • Why is this approach appealing? • Doesn’t require apps to be modified • Hints not limited to specific access heuristics • However … • Speculation has to be correct despite fake data • Speculation has to be lightweight

  9. Spec. execution approach

  10. Speculative execution • Assign each process a speculative thread • Run speculative thread when original stalls • Speculative thread generates I/O hints • Hints used by storage system to pre-fetch data • In what address space should thread run? • Needs access to process state • But what shouldn’t spec. thread be able to do? • Modify process state • Should have no direct side-effects on original’s state

  11. Speculative execution • Ways that the spec. thread could cause side-effects • Could modify data in address space • Could modify data outside address space (e.g., files) • Could generate signals (e.g., seg fault) • How do would you prevent this? • Use software-enforced copy-on-write • Instrument loads/stores and point to copy versions • But won’t instrumentation degrade performance? • Make a copy of the program text • Only instrument copy • Original program can run un-instrumented

  12. Speculative execution • When speculation goes “off track” • Speculative thread operates on fake data • Fake data could affect program control flow • Could cause speculative thread to execute differently than orig. • Is there any way to prevent this? • Not really • If we knew content of real data would return it • Have to detect off track speculation instead • How should we detect off-track speculation? • Maintain a log of hints • Original thread checks if hints are correct • If they differ, then speculation has gone off track

  13. Speculative execution • When speculative execution is effective • Speculative thread can overlap with slow operation • Speculative thread is much faster than slow operation • Speculation rarely goes off track • i.e., fake data must not meaningfully affect control flow • i.e., result of slow operation is often known in advance • Other slow operations besides disk reads? • Distributed consistency protocols (two-phase commit) • Synchronous meta-data writes • Lots of interesting uses for speculative execution

  14. Speculative execution • Why are multi-threaded apps hard? • Have to ensure that thread interleavings are same • On a uni-processor possible most of the time • On a multi-core machine much harder … • Area of active research

  15. Does it work? Hints are really useful Generating hints is cheap Runtime with spec + hints Runtime with spec + no hints

More Related