RADAR: Dataflow Analysis for Concurrent Programs using Datarace Detection

1. RADAR: Dataflow Analysis for Concurrent Programs using Datarace Detection Ravi Chugh, Jan Voung, Ranjit Jhala, Sorin Lerner {rchugh, jvoung, jhala, lerner} @ cs.ucsd.edu UC San Diego

2. Studying Concurrency is Important

3. Studying Concurrency is Important

4. Studying Concurrency is Important

5. Studying Concurrency is Important • A “counting argument” • > wget http://pldi2008.cs.ucr.edu/program_schedule.html • > grep –i “concurr” program_schedule.html | wc –l • 6

6. All Jokes Aside… Architectural Trends Pressing Issue Devilish Bugs

7. Our Approach Leverage sequential work Sequential Dataflow Analysis Concurrent Dataflow Analysis

8. Sequential Reasoning is Incorrect p = new_or_die(); p != null x = compute(); p != null *p = x; if (p != null) { use(p); p = null; }

9. Sequential Optimization is Incorrect flag = 0; while (flag == 0) { /* spin */ } use(g); Constant propagation would be incorrect g = compute(); flag = 1; 0 • Compilers forced to be conservative • Optimization opportunities missed

10. Our Approach Concurrent Dataflow Analysis RADAR Sequential Dataflow Analysis

11. Our Approach Sequential Dataflow Analysis Concurrent Dataflow Analysis RADAR Concurrency Analysis Race Detector

12. Modular Framework Sequential Non-Null Analysis Sequential Constant Analysis Sequential Dataflow Analysis Concurrent Dataflow Analysis Concurrent Non-Null Analysis Concurrent Constant Analysis RADAR Race Detector

13. Modular Framework Sequential Constant Analysis RADAR Concurrent Constant Analysis Race Detector precise scalable

14. RADAR: Results • Concurrent non-null analysis • Scales to: • Apache (~130 KLOC) • OpenSSL (~210 KLOC) • subset of Linux kernel (~830 KLOC)

15. Example with Locks lock(l); p = new_or_die(); x = compute(); *p = x; unlock(l); lock(l); if (p != null) { use(p); p = null; } unlock(l);

16. Can this be Optimized? lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l); lock(l); if (p != null) { use(p); p = null; } unlock(l);

17. Optimized Version is Race-free lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l); lock(l); if (p != null) { use(p); p = null; } unlock(l);

18. Oops… lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l); lock(l); if (p != null) { use(p); p = null; } unlock(l);

19. Sequential Non-Null Analysis lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); lock(l); if (p != null) { p != null use(p); p != null p = null; p != null } unlock(l);

20. RADAR on Example lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); lock(l); if (p != null) { p != null use(p); p != null p = null; p != null } unlock(l);

21. RADAR on Example lock(l); p = new_or_die(); p != null unlock(l); x = compute(); lock(l); *p = x; unlock(l); Can fact be invalidated by concurrent thread? Can p be written by a concurrent thread?

22. RADAR on Example lock(l); p = new_or_die(); p != null unlock(l); x = compute(); lock(l); *p = x; unlock(l); Race Detector No Race pseudo-read(p) Race Can p be written by a concurrent thread?

23. RADAR on Example lock(l); p = new_or_die(); p != null unlock(l); x = compute(); lock(l); *p = x; unlock(l); Race Detector No Race pseudo-read(p) Race Can p be written by a concurrent thread?

24. RADAR on Example lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); lock(l); *p = x; unlock(l); Race Detector No Race Race pseudo-read(p)

25. RADAR on Example lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; unlock(l); UNSAFE

26. RADAR on Safe Example lock(l); p = new_or_die(); p != null x = compute(); *p = x; unlock(l); Race Detector No Race pseudo-read(p) Race

27. RADAR on Safe Example lock(l); p = new_or_die(); p != null x = compute(); p != null *p = x; unlock(l); Race Detector No Race Race pseudo-read(p)

28. RADAR on Safe Example lock(l); p = new_or_die(); p != null x = compute(); p != null *p = x; unlock(l); SAFE

29. More on RADAR • Round trip queries to race detector • Inter-procedural analysis

30. (1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); Race Detector No Race pseudo-read(p) Race Possible Race pseudo-read(p) Allow sequential analysis to run Get superset of (concurrent) facts

31. (1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); Insert all pseudo-reads at once pseudo-read(p) Race Detector pseudo-read(p) pseudo-read(p) pseudo-read(p) Send whole program to race detector pseudo-read(p)

32. (1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; p != null unlock(l); pseudo-read(p) Get results back from Race Detector pseudo-read(p) pseudo-read(p) pseudo-read(p) pseudo-read(p)

33. (1) Round-trip Queries lock(l); p = new_or_die(); unlock(l); x = compute(); lock(l); *p = x; unlock(l); pseudo-read(p) Get results back from Race Detector pseudo-read(p) pseudo-read(p) Rerun analysis using race results pseudo-read(p) pseudo-read(p)

34. (1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); lock(l); *p = x; unlock(l); Rerun analysis using race results

35. (1) Round-trip Queries lock(l); p = new_or_die(); p != null unlock(l); p != null x = compute(); p != null lock(l); p != null *p = x; unlock(l); UNSAFE

36. (2) Handling Procedures void foo() { if (*) { unlock(l); compute(); lock(l); } } lock(l); if (p != null) { foo(); *p = 10; } unlock(l); lock(l); p = null; unlock(l); Unlock in foo allows interference Want to summarize effect of calling foo

37. (2) Handling Procedures void foo() { if (*) { unlock(l); compute(); lock(l); } } lock(l); if (p != null) { foo(); *p = 10; } unlock(l); lock(l); p = null; unlock(l);

38. (2) Handling Procedures lock(l); if (p != null) { foo(); *p = 10; } unlock(l); pseudo-unlock(l); pseudo-read(p); pseudo-lock(l); • In the caller, RADAR inserts: • pseudo-unlock for every unlock in foo • pseudo-reads

39. Evaluation • Is RADAR scalable? • Is RADAR precise? • Where can we do better?

40. Experiments: Benchmarks • Apache 2.2.6 (130 KLOC) • worker threads + modules (e.g., caches) • OpenSSL 0.9.8g (210 KLOC) • model a multi-threaded client • Linux 2.6.15 (830 KLOC) • subset from RELAY experiments

41. Sequential Race Detector + = Non-Null Escapes ⇒ race Conservative Non-Null Non-Null Optimistic (sequential) Never race Non-Null

42. GAP Sequential Race Detector + = Non-Null Escapes ⇒ race Conservative Shared ⇒ race Non-Null No locks ⇒race Non-Null Optimistic (sequential) Never race Non-Null

43. Sequential Race Detector + = Non-Null Escapes ⇒ race Conservative Shared ⇒ race Non-Null No locks ⇒race Non-Null Optimistic (sequential) Never race Non-Null

44. Sources of Imprecision • Alias analysis • affects sequential dataflow and race detection • Lockset-based race analysis • ignores fork, join, condition variables • RADAR framework • pseudo-read (for non-null fact) races with • “x = NonNullAddress;”

45. Related Work • Programming-model-based approaches • [Knoop et al 96], [Grunwald et al 93], … • par-begin / par-end • handles introduction of facts between threads • Thread-modular • [Owicki et al 76], [Jones 83], [Flanagan et al 03], … • more precise (use environment assumption) • inference not as scalable

46. Conclusion Sequential Dataflow Analysis RADAR Concurrent Dataflow Analysis Race Detector precise scalable

47. THANKS!

49. What is filtered for non-null? if (a->f != null) { b->f = null; deref(a->f); //filter if warned deref(b->f); //don’t filter }