Tools for the development of parallel applications

1. Tools for the development of parallel applications Assaf Schuster Technion – Israel Institute of Technology Intel MultiCore Conference

2. Message Passing? • People will not give up shared memory programming • Do not want to learn messaging APIs • Shared memory considered a “natural” extension of serial code • Distributed computing sounds scary; multithreading? “I’ve tried it, it worked ” • Shared memory considered “easier to implement” on upper layers of memory hierarchy Intel MultiCore Conference

3. Multithreading Issue #1:Debugging Tools The hard problems: over- and under-synchronization • Very few tools • No full coverage • No precise tools • No scalable tools • No educational material • Some positive research techniques – very limited Intel MultiCore Conference

4. Therac 25 • A medical radiation machine to treat cancer • 6 patients got a radiation overdose • 4 died • 2 injured Intel MultiCore Conference

5. State of the Art - Industry Intel Thread Checker? IBM’s Contest: • Add yields on random locations • Expose deserted interleavings • Watch the output: any problem? • Look for the causes of the problem in the interleaving Steps 2 & 3 human-based  “travel debugging” Intel MultiCore Conference

6. State of the Art – Research On the fly DR detection for C++ programs Instrumentation-based [Pozniansky and Schuster, PPoPP 2003](4-way IBM Netfinity server, 550MHz, Win-NT)

7. Playing With Detection Granularity Coverage: a single interleaving [Pozniansky and Schuster, PPoPP 2003](4-way IBM Netfinity server, 550MHz, Win-NT)

8. State of the Art - Research • Model Checking DRs (IBM tools) • Boosted with Dynamic Information (Lockset) • Provides a witness to the race • Detects races in many interleavings • …….. • Scalability: 4 threads max • Scalability: Sequential Consistency only [Sagiv, Shacham, Schuster, PPoPP 2005] Intel MultiCore Conference

9. Multithreading Issue #2:Memory Models • Observation: most current multithreaded “scalable” applications were/are created by programmers who do not know what MM is all about. • No educational material • “Here’s my new great book: introduction to parallel programming…” • Existing debugging tools ignore MM Intel MultiCore Conference

10. Memory Models Education T1 T2 ======= ======= Initially X=Y=0 Print(X) Print(Y) Y = 1 X = 1 Is <<< 1 1 >>> a legitimate output? Intel MultiCore Conference

11. Java Memory Model • The first popular language attempting a “formal definition”. JLS 1995, 17 pages. • Theorem: JMM is Coherent[Gontmakher and Schuster, IPDPS 1997] • Did they know? • Consequence: ***All*** tested JVMs in 1999-2000 breach JMM specification. Intel MultiCore Conference

12. p and q might point to same object p.x = 0 a=p.x p.x=1 b=q.x reads can make a process see writes by another process. The read “kills” later reuse of local values. c=p.x Cannot put c=a assert(p==q  a  b  c) Coherence makes reads preventcommon compiler optimizations Intel MultiCore Conference

13. August 2004: A New JMM • Java Community Process, JSR-133 • Bill Pugh • Sarita Adve • And others • Took three years (could have been longer) • Is now being adopted as a standard for C++ • But is it any good?????? Intel MultiCore Conference

14. The program is correctly synchronized if it is data-race free on a sequentially consistent platform Java must guarantee sequentially consistent behavior for correctly synchronized program Causality requirements: guarantees for correctly synchronized programs Initially, x = y = 0 Thread 1 | Thread 2 r1 = x; r2 = y; if (r1 != 0) if (r2 != 0) y = 1; x = 1; r1 == r2 == 0 is the only legal behavior Intel MultiCore Conference

15. Java must provide safety guarantees for incorrectly synchronized programs: values must not appear “out of thin air” Initially, x = y = 0 Thread 1 | Thread 2 r1 = x; r2 = y; y = r1; x = r2; Incorrectly synchronized. Still: r1 == r2 == 42 must not be allowed Causality Requirements: Safety Guarantees 42 42 42 42 4242 42 42 4242 42 42 4242 justified Intel MultiCore Conference

16. Causality Test Case 6 Initially, A = B = 0 Thread 1 | Thread 2 1: r1 = A; | 3: r2 = B; if (r1 == 1) | if (r2 == 1) 2: B = 1; | 4: A = 1; | if (r2 == 0) | 5: A = 1; r1 == r2 == 1 ALLOWED?! Intel MultiCore Conference

17. Conclusions • Parallelization algorithms, load balancing, most of the work in 30 years -- usually non issues • Debugging tools for multithreading • We do not have good technology, yet • And may never have, a hard problem • Memory models for shared memory • We do not know what we want, yet • I do not see programmers understand them • Solution • Better programming paradigms • Dara-race free languages • Valiant’s BSP (shared memory-like, easy to debug) • Next panel! Intel MultiCore Conference