Asserting and Checking Determinism for Parallel Programs

1. Asserting and Checking Determinismfor Parallel Programs Jacob Burnim KoushikSen* EECS, UC Berkeley

2. Motivation • Parallel programming is difficult • Culprit: Non-determinism • Interleaving of parallel threads • But required to harness parallelism • Sequential programs produce deterministic results • To make parallel programming easy • We want determinism • Same input => semantically same output.

3. Determinism Efforts • Language design [DPJ, X10,Yada] • Deterministic by design: Types and annotations • Constrains programming • May need non-determinism for better performance • Deterministic runtime [DMP, Kendo] • Race detection • Absence of data races does not imply determinism • Data races could be benign and could help in improving performance • Determinism Checker [SingleTrack, Torrellas et al.] • Dynamic determinism checker. • Determinism with respect to abstraction [Rinard et al.]

4. Our Goal • Provide a framework that can express relevant determinism directly and easily • Separate from functional correctness specification • Allow data races and healthy non-determinism • Language independent • Useful for QA

5. Our Goal • How to specify correctness of parallelism? Implicit: No sourcesofnon-determinism(no data races) Explicit: Full functional correctness. • Determinism specification: A sweet spot? • Lightweight, but precise.

6. Outline • Motivation • Deterministic Specification [FSE’09] • Checking: Active Testing • Experimental Evaluation • Future Work + Conclusions

7. Deterministic Specification • Goal: Specify deterministic behavior. • Same initial parameters => same image. • Non-determinism is internal. // Parallel fractal render mandelbrot(params, img);

8. Deterministic Specification • Program: , Initial State: , Schedules: • Specifies: Two runs from same initial program state have same result state for any pair of schedules deterministic { // Parallel fractal render mandelbrot(params, img); }

9. Deterministic Specification • Too restrictive – different schedules may give slightly different floating-point results. double A[][], b[], x[]; ... deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); }

10. Deterministic Specification • Too restrictive – internal structure of set may differ depending on order of adds. set t = new RedBlackTreeSet(); deterministic { t.add(3) || t.add(5); }

11. Deterministic Specification • Too restrictive – search can correctly return any tree with optimal cost. deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); }

12. Semantic Determinism • Too strict to require every interleaving to give exact same program state: deterministic { P }

13. Semantic Determinism • Too strict to require every interleaving to give exact same program state: deterministic { P } Predicate! Should be user-defined.

14. Semantic Determinism • Too strict to require every interleaving to give exact same program state: • Specifies: Final states are equivalent. deterministic { P } assert Post(s1,s1’)

15. Semantic Determinism double A[][], b[], x[]; ... deterministic { // Solve A*x = b in parallel lufact_solve(A, b, x); } assert (|x – x’| < ε) “Bridge” predicate

16. Semantic Determinism • Resulting sets are semantically equal. set t = new RedBlackTreeSet(); deterministic { t.add(3) || t.add(5); } assert (t.equals(t’))

17. Semantic Determinism deterministic { // Parallel branch-and-bound Tree t = min_phylo_tree(data); } assert (t.cost == t’.cost())

18. Preconditions for Determinism set t = … deterministic { t.add(3) || t.add(5); } assert (t.equals(t’)) … deterministic { t.add(4) || t.add(6); } assert (t.equals(t’)) • Too strict – initial states must be identical • Not compositional.

19. Preconditions for Determinism • Too strict to require identical initial states: deterministic { P } assert Post(s1,s1’)

20. Preconditions for Determinism • Too strict to require identical initial states: deterministic assume (s0 = s0’) { P } assert Post(s1,s1’)

21. Preconditions for Determinism • Too strict to require identical initial states: deterministic assume (s0 = s0’) { P } assert Post(s1,s1’) Predicate! Should be user-defined.

22. Preconditions for Determinism • Too strict to require identical initial states: • Specifies: deterministic assume Pre(s0,s0’) { P } assert Post(s1,s1’)

23. Bridge predicates/assertions deterministic assume Pre(s0,s0’) { P } assert Post(s1,s1’) “Bridge”predicate “Bridge”assertion

24. Preconditions for Determinism • Specifies: Semantically equal sets yield semantically equal sets. set t = ... deterministic assume (t.equals(t’) { t.add(4) || t.add(6); } assert (t.equals(t’))

25. Advantage • Separately check parallelism and functional correctness. • Show parallelism is outwardly deterministic. • Reason about correctness sequentially. • Decompose correctness proof! • Example: • Write Cilk program and prove (or test) sequential correctness. • Add parallelism, answers should not change

26. Determinism vs. Atomicity • Internal vs. external parallelism/non-determinism • Complementary notions Deterministic Atomic “Closed program” “Open program”

27. Other Testing Use sequential program as spec deterministic Pre(s0,s0’) { if (*) { SequentialPgm; } else { ParallelPgm; } } assert Post(s,s’); Regression testing deterministic Pre(s0,s0’) { if (*) { PgmVersion1; } else { PgmVersion2; } } assert Post(s,s’);

28. Outline • Motivation • Deterministic Specification [FSE’09] • Checking: Active Testing [PLDI 08,09, FSE 08, CAV 09] • Experimental Evaluation • Future Work + Conclusions

29. Checking: Active Testing Predicting and Exploring “interesting” Schedules

30. Active Testing:Predict and Test Potential Bugs • Predict potential bugs: • Data races: Eraser or lockset based • Atomicity violations: cycle in transactions and happens-before relation • Deadlocks: cycle in resource acquisition graph • Memory model bugs: cycle in happens-before relation • Test schedules to create those bugs [ASE 07, PLDI 08, FSE 08, PLDI 09, FSE 09, CAV 09]

31. Active Testing Cartoon: Phase I Potential Collision 2 1 1 3 2

32. Active Testing Cartoon: Phase II 2 1 1 3 2

33. Outline • Motivation • Deterministic Specification • Checking: Active Testing • Experimental Evaluation • Future Work + Conclusions

34. Ease of Asserting Determinism • Implemented a deterministic assertion library for Java. • Manually added deterministic assertions to 13 Java benchmarks with 200 – 4k LoC • Typically 5-10 minutes per benchmark • Functional correctness very difficult.

35. Ease of Use: Example Deterministic.open(); Predicate eq = new Equals(); Deterministic.assume(width, eq); … (9 parameters total) … Deterministic.assume(gamma, eq); // Compute fractal in threads int matrix[][] = …; Deterministic.assert(matrix, eq); Deterministic.close();

36. Effectiveness in Finding Bugs • 13 Java benchmarks of 200 – 4k LoC • Ran benchmarks on ~100 schedules • Schedules with data races and other “interesting” interleavings (active testing) • For every pair of executions ofdeterministic Pre { P } Post:check that:

37. Experiments: Java Grande Forum

38. Experiments: Parallel Java Lib

39. Experimental Evaluation • Across 13 benchmarks: • Found 40 data races. • 1 violates deterministic assertions.

40. Experimental Evaluation • Across 13 benchmarks: • Found 40 data races. • 1 violates deterministic assertions. • Found many interesting interleavings(non-atomic methods, lock races, etc.) • 1 violates deterministic assertions.

41. Determinism Violation deterministic { // N trials in parallel. foreach (n = 0; n < N; n++) { x = Random.nextDouble(); y = Random.nextDouble(); … } } assert (|pi - pi’| < 1e-10) • Pair of calls to nextDouble() mustbe atomic.

42. Outline • Motivation • Deterministic Specification • Checking: Active Testing • Experimental Evaluation • Future Work + Conclusions

43. How to verify: Essential Rule deterministic assume(Φ1) A11;A12;A13|| A21;A22;A23|| A31;A32;A33 } assert (Φ2)

44. How to verify: Essential Rule deterministic assume(Φ1) A11;A12;A13|| A21;A22;A23|| A31;A32;A33 } assert (Φ2) Find an invariant Φ and prove that for all i, j, l, m deterministic assume(Φ) Aij|| Alm } assert (Φ) and Φ1 → Φ and Φ → Φ2

45. Prove for the following deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } } assert(best=best’)

46. Decompose the proof Let us prove that P ≈ S That is prove that deterministic assume(Pre(S0,S’0)) { if (*) P else S } assert (Post(S,S’))

47. Decompose the proof Let us prove that P ≈ S Create a non deterministic sequential program NS [Galois] P ≡ NS ≈ S

48. Decompose the proof Let us prove that P ≈ S Create a non deterministic sequential program NS P ≡ NS ≈ S deterministic assume(S0=S’0) { if (*) P else NS } assert (S=S’) deterministic assume(Pre(S0,S’0)) { if (*) NS else S } assert (Post(S,S’))

49. How to verify? deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } } assert(best=best’) Not deterministic sequential program while (!wq.is_empty()) { [work = wq.nondet_get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } }

50. How to verify? Not deterministic sequential program while (!wq.is_empty()) { [work = wq.nondet_get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } deterministic assume(true) { parallel while (!wq.is_empty()) { [work = wq.get();] if (bound(work) >= best) continue; if (size(work) <= threshold) { s = find_best_solution(work); [best = min(best,cost(s));] } else { [wq.add_all(split(work));] } } } assert(best=best’) Barrier()