Formal Design and Verification Methods for Shared Memory Systems

1. Formal Design and Verification Methods for Shared Memory Systems Ratan Nalumasu Dissertation Defense September 10, 1998

2. Problems Facing Digital Design • Complexity • Longer design time • Shorter time to market Design Complexity

3. Current Debugging Technology • Full model • Partial examination  No assurance • Weaker properties • Difficult correctness metrics • Full model Design Complexity

4. Formal Methods • Formal methods = Math based techniques • Continuous math : Engineering = • Discrete math : Digital system design • “It is what the designers want. It’s just challenging to prove.” Introduction to FM

5. Formal Methods based Design • Reduced model • Complete examination • Better assurances (on the reduced model) • Stronger property language • Better correctness metrics • Reduced model Introduction to FM

6. FM Taxonomy • Manual verification techniques: Interactive theorem provers • Automatic verification techniques: Model checkers • Compilation techniques: • Refinement rules Introduction to FM

7. Interactive Theorem Provers • Can deal with infinite state systems • Extensive manual reasoning • Good for algorithm verification Proof of a compilation scheme Theorem Provers

8. (G=0, p.L=0, ...) 0 1 3 2 Model Checking process p(x) { global G; local L; while (...) { recv ...; send ...; } } process q(x,y) ... Model Checking

9. Model Checking Strengths • Automatic • If property fails, model checker shows the error trace • Deadlock: How initial state reached it • Assertion: How initial state reached it • Starvation: A loop where no progress is made Model Checking

10. 00 0 0 1 2 10 01 1 11 02 20 12 2 21 22 Model Checking: Example • Construct graph of the system, and check the property: Deadlock at (22) • State Explosion Partial Order Reductions Model Checking

11. Refinement Algorithms • Need to verify only high-level protocols • Domain-specific compilers can generate efficient implementations Refinement rules for DSM protocols Refinement Algorithms

12. State of the art of Applied FM • General purpose • Widely applicable techniques • Inefficient algorithms • Inefficient “compilers” • Do not help with domain specific concerns Applied FM

13. CPU CPU Domain: Memory Memory Thesis Statement • Domain specific formal methods • Efficient verification techniques • Address domain specific concerns

14. Overview • Introduction to formal verification • Shared memory systems • Contributions • Conclusions

15. Memory Bottleneck • Processor speed increases at 55% a year, while memory speed increases at 7% • Caches • Tendency toward multiprocessors • Further imbalance  complex protocols • SMP systems • DSM systems Memory Bottleneck

16. CPU $ CPU $ CPU $ Memory Symmetric Multiprocessors Can scale upto 10s of processors Modern caches have support for such SMP protocols SMP Architecture

17. SMP Protocol Design • Bus protocols • Bus arbitration algorithm • Cache invalidation scheme • Lack of atomicity on the bus • Bus and CPU interaction • Does CPU have out-of-order execution? • Does bus allow out-of-order completion? • Are these decisions visible to software? SMP Protocols

18. MEM MEM MEM NODE NODE NODE Network Distributed Shared Memory Each node may be a SMP or a single CPU DSM Architecture

19. DSM Protocol Design • Network port arbitration • Coherency maintenance across the network • Maintaining distributed state • Little atomicity • “Ghost” messages • Transient states • Are these decisions visible to software? DSM Protocols

20. Shared Memory Correctness • Low level: • deadlock • forward progress • bus arbitration • Intermediate level: • at most one owner of a cache line at a time • High-level: • abstraction provided to the software Shared Memory Systems

21. P1 read(b) write(a,new) Uniprocessor: P1 write(a,new) read(b) ok cache/compiler/ out-of-order execution P1 write(a,new) read(b) P2 write(b,new) read(a) Multiprocessor: Not ok under S.C. P2 read(a) write(b, new) P1 read(b) write(a,new) Abstraction Provided to Software Test model checking Software Interface

22. Overview • Introduction to formal verification • Shared Memory systems • Contributions • mitigating state explosion • Partial order reduction algorithm • facilitating high-level design • Protocol synthesis algorithm • enhancing applicability • High-level correctness such as SC • Conclusions

23. 2 Test Model checking 2 1 PO algorithm 3 Refinement rules 3 Efficient implementation Contributions Protocol Contributions

24. Contribution #1 Mitigating State Explosion Problem Partial Order Reductions

25. 00 10 01 11 02 20 0 00 0 1 2 12 21 10 1 22 20 2 21 22 Partial Order Reductions If two transitions are independent, then explore one of them postponing the other PO Reductions

26. S0 Postponed S1 Postponed Ignoring Problem • Select some transitions, and postpone others  but do not postpone forever PO Reductions

27. S0 Expand Proviso based Solution • Godefroid, Valmari, Holzmann, Peled’s solutions are very similar: Proviso • Expands the “last” state of the loop completely Postponed S1 PO Reductions

28. 0 0 1 P 2 1 Q 2 20 12 11 01 10 00 22 02 21 Q postponed Problem with Proviso ALL 9 states PO Reductions

29. 00 01 10 20 02 Our Algorithm: 2-phase 0 0 1 P 2 1 Q 2 Only 5 states PO Reductions

30. (20x) Performance Comparison PO Reductions

31. Contribution #2 Facilitating High-level Design Protocol Refinement

32. Protocol Refinement • PO reductions not sufficient, theorem provers ruled out • Compile from high-level protocol specification • easier to design • easier to verify • can generate efficient implementation using domain knowledge Refinement Algorithms

33. ??? Some request Unexpected Messages Send a req to Q recv ack from Q P Always nack no forward progress Always Silence  Deadlock Refinement Algorithms

34. Refinement Procedures • Debug the high-level specification: Synchronous communication with no transient states • Automatic refinement procedures transforms it into detailed implementation • No need to verify the implementation • Needs domain specific knowledge for efficiency Refinement Algorithms

35. Related Work • Buckley & Silberschatz, 83 • For OS environments, not fit for hardware • Gribomont,90 • Protocols where synchronous messages can be simply replaced by asynchronous messages Refinement Algorithms

36. Related Work (contd) • Teapot, 96 for DSM systems (Chandra) • Protocol programming language • “Suspend” construct for transient states • Not high-level: Suspend states still specify what to do in a transient state Refinement Algorithms

37. Network Context: DSM Protocols MEM MEM MEM NODE NODE NODE Protocol per each cache line 1 home, n “remote” nodes per each line Home is responsible for maintaining consistency (Hub) Refinement Algorithms

38. Req Req Ack or Nack Ack or Nack Home Remote Home Remote Refinement Rules Refinement Algorithms

39. Refinement Rules (2) Req1 Req2 Req1 is ignored by both processes Ack or Nack Home Remote Refinement Algorithms

40. Debugging Effort Protocol compilation scheme has been proved using a theorem prover Refinement Algorithms

41. Contribution #3 Enhancing Applicability Shared Memory Model Verification

42. P1 write(a,new) read(b) P2 write(b,new) read(a) P1 read(b) write(a,new) P2 read(a) write(b,new) Under SC Relaxing Instruction Orders Test Model Checking

43. Test model checking Verification of HW/SW Interface CPU $ CPU $ CPU $ Memory SC: The result can be explained by some interleaving of the instructions. Test Model Checking

44. Current Verification Techniques • Simulation • Must study lengthy executions • Must choose non-trivial programs • Formal techniques (next slide) Test Model Checking

45. Related Work • Graf’s Lazy caching in ACTL* • Gibbons approach  run programs and check if the results are SC • McMillan’s thesis  data abstraction for a test • Hojati  data abstraction in a different context • Undecidability result by Alur et al Test Model Checking

46. ACTL* for (stronger than) SC • AG(enabled( read(a,d) ))  avail(a,d) • AG(avail(a,d) AND EF(enable(read(a,d)))) A[NOT avail(a,d) W AG NOT avail(a,d)] • ... • init  AG[after(write(a,d))  A(NOT enabled(read(a,d) W avail(a,d))] • Such MODEL DEPENDENT SPECS do not fit in an iterative industrial frame Test Model Checking

47. Test Model Checking • Adaptation of simulation to model checking • model checking (full coverage) + • testing (“black box approach’’) • Tests are independent of the model being verified  manual effort is considerably reduced • Test model-checking can be used early in the design cycle Test Model Checking

48. Results • Defined a shared memory description language • “data is not used for control decisions” • “addresses are symmetric” • Can specify HP’s Runway/PA, ... • Model checking technique • “Small number of addresses is sufficient” • Application to runway/PA using PV Test Model Checking

49. X(i+1)  X(i) Read Order, Write Order If P1 executes two write instructions, then P2 sees them in the program order of P1 P1 A := 1 A := 2 A := 3 .... A := k P2 X1 := A X2 := A X3 := A .... Xk := A Many deficiencies Test Model Checking

50. Deficiencies of the Test • Finite k • What if an error occurs for a really large k? • Location “A” is never written by P2 • What if an error occurs when the ownership changes? • Only 1-address • The definitions of RO and WO are not restricted to a single address at a time • How many addresses to consider? Test Model Checking