1 / 20

Language extensions for speculative parallelism

Language extensions for speculative parallelism. Kapil Vaswani (Microsoft Research, India) Prakash Prabhu (Princeton University) G. Ramalingam (Microsoft Research, India). Speculation. Take a risk in anticipation of gain Widely used to optimize performance Caching and pre-fetching

obelia
Download Presentation

Language extensions for speculative parallelism

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Language extensions for speculative parallelism Kapil Vaswani (Microsoft Research, India) Prakash Prabhu (Princeton University) G. Ramalingam (Microsoft Research, India)

  2. Speculation • Take a risk in anticipation of gain • Widely used to optimize performance • Caching and pre-fetching • Branch prediction, speculative code motion in compilers • Speculative parallelism • Doing work in parallel anticipating later use • Software transactions and futures • Focus of this talk • Value speculation based parallelization

  3. Value speculation Producer Consumer value T Consumer Producer Prediction function Correct value T = Consumer Speculative value T’

  4. Lexical analysis <html><p><a href=“http://research.microsoft.com”>Microsoft Research</a></p></html> h Accounts for a large fraction of execution time in browsers, word processing apps < > t m l Lexical analyzer <html>, <p>, <a, href, =, http://research.microsoft.com, >, Microsoft Research, </a>, </p>, </html>

  5. Huffman decoding 1.00 0 1 010000011110010… 01000011110010… 0.60 0.40 00 01 10 11 Huffman decoder 0.30 d, 0.20 a, 0.30 010 011 e, 0.20 baaeead… c, 0.15 b, 0.15 pos = 0; while (pos < inputBuffer.size) { bitsRead = decode(inputBuffer, pos); pos = pos + bitsRead; }

  6. What’s in common? • “Sequential” loops • Cross iteration dependencies • Not directly amenable parallelization • Not hard to find  • Game playing • Planning and scheduling • Data flow analysis • Dynamic programming • Machine learning

  7. Speculative Huffman decoding 0011000010… 0110011100… … k ith chunk A A E E A D .… 0 1 0 0 0 0 1 1 1 1 0 0 1 0 0 1 0 0 0 0 1 1 1 1 0 0 1 0 B A C E A D

  8. Speculative lexical analysis (Bodik et al) <HTML><p>… … … </HTML> s1 k ith chunk s1 sik k s1

  9. Performance

  10. Language extensions

  11. Why language extensions? • Careful speculation requires thought • Prediction functions domain specific • Not always possible to automate • Writing correct speculative code non-trivial • Spawning and scheduling threads • Check predictions, cancel and re-invoke consumers • Check conflicts between producers and consumers • Deal with exceptions • Let programmer specify speculation

  12. Speculative parallel for Parallel.For (int from, int to, void delegate(Int32) work) Speculation.Parallel.For(int from, int to, T delegate (Int32, T) work, T delegate (Int32) predictor)

  13. Speculative parallel for from from + 1 from + 2 to Original loop work work work work T T T … from from + 1 from + 2 to pred pred pred pred … Speculative parallel for T T T T work work work work … = T T T T

  14. Example: speculative lexical analysis int overlap = 20; intchunk_size = total_size / p; Speculation.Parallel.For(0, p, (i, state) => { return LexicalAnalysis(i*chunk_size, (i + 1) * chunk_size, state); }, (i) => { if (i == 0) return START_STATE; else return LexicalAnalysis( (i – 1) * chunk_size – overlap, (i – 1) * chunk_size, START_STATE); ) } ); Loop iteration Prediction function

  15. Speculative parallel for • Thread creation and scheduling • Check predictions • If incorrect, cancels and re-run next iteration • One misprediction does not imply that all subsequent iterations are wrong! • Deal with speculative exceptions

  16. Safety • Safe variant • Guarantees sequential semantics • Increased sequential cost • Unsafe variant • Responsibility of managing side-effects left to the programmer • A type system that guarantees safety

  17. Speculative task Stand-alone speculative construct

  18. Example: speculative DFS intSpeculativeDFS (Heap h, int position, intsearchValue) { if (h.ValueAt(position) == searchValue) return position; Speculate ( () => { return SpeculativeDFS(h, 2 * position + 1, searchValue); }, (found) => { if (found != NOT_FOUND) return found; else return SpeculativeDFS(h, 2 * position + 2, searchValue); } () => { return NOT_FOUND; } ); } Producer Consumer Prediction function

  19. Summary • Speculation is an interesting algorithm design and programming idiom • A library that supports speculative constructs • On the lookout for more speculative patterns and idioms

  20. Questions?

More Related