Intel Concurrent Collections (for Haskell)

1. IntelConcurrent Collections (for Haskell) Ryan Newton*, Chih-Ping Chen*, Simon Marlow+ *Intel +Microsoft Research Software and Services Group Jul 29, 2010

2. C C n Goals Streaming/Actors/ Dataflow • Push Parallel Scaling in Haskell • Never trivial • Harder with: Laziness, complex runtime • Offer another flavor of parallelism: • Explicit graph based a`par`b forkIO$do… Pure operand parallelism Threaded parallelism Data Parallelism [:x*y|x<-xs|y<-ys:]

3. C C n Intel Concurrent Collections (CnC) Streaming/Actors/ Dataflow • These models: A bit more domain specific (than, say, futures) • A payoff in parallelism achieved / effort • CnC is approximately: • stream processing + shared data structures • Intel CnC: A library for C++ • Also for Java, .NET thanks to Rice U. (VivekSarkar et al) • Yet… CnC requires and rewards pure functions -- Haskell is a natural fit.

4. Goals How would I (typically) do message passing in Haskell? • Push Parallel Scaling in Haskell • Never trivial • Harder with: Laziness, complex runtime • Offer another flavor of parallelism: • Explicit graph based a`par`b forkIO$do… Pure operand parallelism Threaded parallelism Data Parallelism [:x*y|x<-xs|y<-ys:]

5. Message passing in Haskell • IO Threads and Channels: do action.. writeChancmsg action.. doaction.. readChanc action.. This form of message passing sacrifices determinism.Let’s try CnC instead…

6. CnC Model of Computation • Tasks not threads • Eager computation, • Takes tag input, • Runs to completion • Step reads data items • Step produces data items • “Cnc Graph” = network of steps • DETERMINISTIC tags items tags items tag 1: 1: ‘a’ 34 2: 2: Step λ tag. Step 3: 3: ‘f’ 4: 4: ‘z’ 99 5: 5: 6: 6: tags There’s a whole separate story about offline analysis of CnC graphs - enables scheduling, GC, etc.No time for it today!

7. How is CnC Purely Functional? Domain Range

8. How is CnC Purely Functional? A set of tag/item collections Updates(new tags, items) MyStep Domain Range

9. AnCnC graph is a function too A set of tag/item collections A complete CnCEval Domain Range

10. Message passing in Haskell • IO Threads and Channels: • Steps, Items, and Tags do action.. writeChancmsg action.. Step1 doaction.. readChanc action.. Step2 This form of message passing sacrifices determinism.Let’s try CnC instead…

11. Haskell / CnC Synergies • Also, Haskell CnC is a fun experiment. • We can try things like idempotent work stealing! • Most imperative threading packages don’t support that. • Can we learn things to bring back to other CnC’s?

12. How is CnC called from Haskell? • Haskell is LAZY • To control par. eval granularity Haskell CnC is mostly eager. • When the result of a CnC graph is needed, the whole graph executes, in parallel, to completion. • (e.g. WHNF = whole graph evaluated) Forkworkers Par exec. Need result! Recv. result

13. A complete CnC program in Haskell myStep items tag =doword1 <- get items "left"     word2 <- get items "right"put items "result" (word1 ++ word2 ++ show tag)cncGraph =dotags <- newTagColitems <- newItemColprescribe tags (myStep items)initialize$doput items "left" "Hello "put items "right" "World "putt tags 99finalize$doget items "result"main = putStrLn (runGraph cncGraph)

14. Initial Results

15. I will show you graphs like this: (4 socket 8 core Westmere)

16. Current Implementations Thread per step instance. • IO based+ pureimpl.s • Lots of fun with schedulers! • Lightweight user threads (3) • Data.Map of MVars for data. • Global work queue (4,5,6,7,10) • Continuations + Work stealing (10,11) • Simple Data.Sequencedeques • Haskell “spark” work stealing (8,9) • (Mechanism used by ‘par’) • Cilkish sync/join on top of IO threads Step(2) Step(1) Step(0) … thr1 thr2 thr3 wrkr1 wrkr1 wrkr2 wrkr3 wrkr2 Globalworkpool Per-threadwork deques

17. Back to graphs…

18. Recent Results

19. Recent Results

20. Recent Results

21. One bit of what we’re up against:Example: Tight non-allocating loops

22. Conclusions from early prototype • Lightweight user threads (3) simple, predictable, but too much overhead • Global work queue (4,5,6,7,10) better scaling than expected • Work stealing (11) will win in the end, need good deques! • Using Haskell “spark” work stealing (8,9) a flop (for now) No clear scheduler winners! CnC mantra - separate tuning (including scheduler selection) from application semantics.

23. Conclusions • Push Scaling in Haskell • Overcame problems (bugs, blackhole issues) • GHC 6.13 does *much* better than 6.12 • Greater than 20X speedups • A start. • GHC will get scalable GC (eventually) • A fix for non-allocating loops may be warranted. • It’s open source (Hackage) - feel free to play the “write a scheduler” game! See my website for the draft paper! (google Ryan Newton)

25. Future Work, Haskell CnC Specific • GHC needs a scalable garbage collector (in progress) • Much incremental scheduler improvement left. • Need more detailed profiling and better understanding of variance • Lazy evaluation and a complex runtime are a challenge!

26. Moving forward, radical optimization • Today CnC is mostly “WYSIWYG” • A step becomes a task (but we can control the scheduler) • User is responsible for granularity • Typical of dynamically scheduled parallelism • CnC is moving to a point where profiling and graph analysis enable significant transformations. • Implemented currently: Hybrid static/dynamic scheduling • Prune space of schedules / granularity choices offline • Future: efficient selection of data structures for collections (next slide) • Vectors (dense) vs. Hashmaps (sparse) • Concurrent vs. nonconcurrent data structures

27. Tag functions: data access analysis • How nice -- a declarative specification of data access!(mystep: i) <- [mydata: i-1], [mydata: i], [mydata: i+1] • Much less pain than traditional loop index analysis • Use this to: • Check program for correctness. • Analyze density, convert data structures • Generate precise garbage collection strategies • Extreme: full understanding of data deps. can replace control dependencies