Operating System Support for Application-Specific Speculation

1. Operating System Support for Application-Specific Speculation Benjamin Wester Peter Chen and Jason Flinn University of Michigan

2. Speculative Execution • Sequential dependent tasks • Predict results of Task Ato break dependence • Execute Task B in parallel • Isolate all effects • Correct prediction: commit • Wrong prediction: abort TIME Predict A A B B EuroSys 2011

3. Speculation Everywhere! • Discrete event simulation • I/O prefetching • Distributed shared memory • Distributed file systems • Deadlock detection • Remote displays • Web page pre-rendering EuroSys 2011

4. Speculation as a Service to Apps How is this system designed? In what ways can it be customized for an app? How can those customizations be specified? EuroSys 2011

5. Outline • Introduction • Designing Speculation as a Service • Implementation • Evaluation • Conclusion EuroSys 2011

6. Design 1: In-App Speculation • Complete semantic info • Predict arbitrary app operations • Safe operations allowed • No reuse: significant development needed • Scope is limited: unsafe operations block App 2 App 1 Data Data EuroSys 2011

7. Design 2: Generic OS Speculation • Apps need no modifications • Wide scope: unsafe operations taint • Lacks semantic understanding of app • Predict system calls only • Handle application conservatively OS App 1 App 2 EuroSys 2011

8. Separate Mechanism and Policy Mechanismimplements isolation Policydescribes customizations Best of both extremes • Mechanism built in OS • Common implementation • Wide scope • Policy specified in Applications • Expose semantic information EuroSys 2011

9. Design 3: Expose Predictions • Predict arbitrary app operations • Reuse OS mechanism(with app assistance) • Wide scope for taint propagation • Limited semantic info • Speculative external output never allowed • Commit on identicalresults OS App 2 App 1 EuroSys 2011

10. Design 4: Expose Safety • Predict arbitrary app operations • Reuse OS mechanism(with app assistance) • Wide scope for taint propagation • More semantic info • Allow safeoutput • Commit on equivalentresults OS App 2 App 1 EuroSys 2011

11. Customizable Policy • Creation • What tasks are predictable • How to predict them • Output • What output is safe to allow • Commit • Which results are acceptable to commit EuroSys 2011

12. Outline • Introduction • Designing Speculation as an OS Service • Implementation • Evaluation • Conclusion EuroSys 2011

13. Implementation • Mechanism built in OS • Based on Speculator kernel • Checkpoints & logs processes, files, IPC, etc. • Policies expressed using system call API EuroSys 2011

14. spec_fork() TIME spec_fork predict A A B Control Process Speculative Process commit abort C B EuroSys 2011

15. API Example int main() { int x; int prediction = get_prediction(); if (spec_fork() == SPECULATIVE) { x = prediction; } else { x = slow_function(); if (equiv(x, prediction)) commit(); else abort(); } set_output_policy(stdout, ALLOW); printf(“%d”, x); } Creation Policy Commit Policy Output Policy EuroSys 2011

16. Outline • Introduction • Designing Speculation as an OS Service • Implementation • Evaluation • Conclusion EuroSys 2011

17. Evaluation Can apps effectively use APIto increase parallelism? Case studies • Predictive application launching in Bash • SSL certificate checks in Firefox • Replicated service in PBFT-CS EuroSys 2011

18. App 1: Predictive Launching in Bash …finished bwester $ ▌ Run program Creation Policy: Use machine learning to predict user input TIME …finished bwester $ grep foo –r . ▌ Check command grep foo –r . Commit Policy: Normalize user input before comparison Output Policy: Safe X11 Messages: Allow EuroSys 2011

19. How Much Work Can Be Hidden? Non-Speculative Speculative 45 Prompt Prompt TIME Spec. Executecmd Get cmd Get cmd Execute Executecmd EuroSys 2011

20. How Much Work Can Be Hidden? Up to 86% hidden Non-Speculative Speculative 45 Prompt Prompt TIME Spec. Executecmd Get cmd Get cmd Execute Executecmd EuroSys 2011

21. App 2: Firefox SSL Connections Creation Policy: Predict certificate is valid Web Server Open https:// TIME Output Policy: Alow SSL connection Check cert. Validation Server Validate Get session key Request page Output Policy: Block private data GET /index.html?id=0123 Done EuroSys 2011

22. Connection Latency Hidden? Non-Speculative Speculative https:// https:// TIME Check cert. Check cert. Session key Session key Validate Request Request Done Done EuroSys 2011

23. Connection Latency Hidden? Non-Speculative Speculative Avg 60ms hidden https:// https:// TIME Check cert. Check cert. Session key Session key Validate Request Request Done Done EuroSys 2011

24. App 3: PBFT-CS Protocol Creation Policy: Agree on 1st reply Request Distributed Replicated Service TIME Send Request (Reply, sig1) Check reply (Reply, sig2) Check reply Work… Output Policy: PBFT-CS Messages: Allow EuroSys 2011

25. Improved Client Throughput? Non-Speculative Speculative TIME Request1 Request1 1st Reply 1st Reply Request2 Null requests 1st Reply Verify Verify Request2 1st Reply EuroSys 2011

26. Improved Client Throughput? 1.9x Throughput Non-Speculative Speculative TIME Request1 Request1 1st Reply 1st Reply Request2 Null requests 1st Reply Verify Verify Request2 1st Reply EuroSys 2011

27. Cost of Generic Mechanism App-specific Mech Shared OS Mech Non-speculative Null requests EuroSys 2011

28. Cost of Generic Mechanism App-specific:8% faster App-specific Mech Shared OS Mech Non-speculative Null requests EuroSys 2011

29. Conclusion • Mechanism • Common: checkpoints, output buffering, taint propagation • Implemented in OS • Policy • App-specific: Controls creation, output, and commit • Implemented in applications • Demonstrated with 3 case studies • Improved parallelism • Small overhead relative to app-specific mechanism Questions? EuroSys 2011