1 / 40

A Coverage Analysis for Safety Property Lists

A Coverage Analysis for Safety Property Lists. Koen Lindström Claessen Chalmers University Gothenburg, Sweden. FMCAD 2007, Austin, Texas. Property-based Verification. Properties. Design. ?. Dynamic Verification. Checkers/ Monitors. PSL/SVA properties. Properties. Design. stimuli.

quinto
Download Presentation

A Coverage Analysis for Safety Property Lists

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Coverage Analysis for Safety Property Lists Koen Lindström Claessen Chalmers University Gothenburg, Sweden FMCAD 2007, Austin, Texas

  2. Property-based Verification Properties Design ?

  3. Dynamic Verification Checkers/ Monitors PSL/SVA properties Properties Design stimuli ? ? ?

  4. Simulation Coverage 95% Statement coverage Path coverage Properties 75% Design In Formal Verification: 100% Gate coverage 98% State space coverage 66% Register coverage 83% stimuli

  5. How About the Properties? 1. A => B 2. B & C => next A 3. not (A and C) 4. next A => B Design ? Removing / adding one of these does not affect coverage…

  6. Property Coverage • We need coverage for properties • How much is the design constrained by properties? • Do the properties express what we want to say? • Are any properties missing? • (lots of work done on this) The more the merrier!

  7. Mutation Coverage Properties Design Design ! ? “Point not covered”

  8. Mutation Coverage • Expensive • Many (failing) calls to model checker • Involves the whole design • Solutions • Symbolically; still expensive • Approximations • Look at proof • Subset of mutations [Chockler, Kupferman,Vardi]

  9. Design may change after analysis This Talk Design does not have to exist • A property coverage analysis • Independent of the design • Finds “forgotten cases” … • … which indicate forgotten properties • Relatively cheap • Only involves properties • Independent of design size • NaN • Not a quantitative analysis There may exist multiple designs

  10. Forgotten Case Analysis 1. prop1 2. prop2 3. prop3 4. prop4 Design

  11. “Forgotten Case”? • Properties specify a function : In  Out? • Too strict • Means complete specification • Synthesizable • Balance • Strict enough: to catch forgotten cases • Loose enough: to be practically usable

  12. In this trace: out1 is not constrained by properties at time 6: Forgotten case! Choice: Forgotten Case Properties OK in1 1 0 1 0 0 0 1 1 1 0 0 … in2 1 0 0 0 1 1 0 0 1 1 0 … in3 0 0 0 1 1 0 0 0 0 1 1 … out1 1 1 0 0 0 1 1 0 1 1 0 … out2 0 1 1 1 0 0 0 0 1 0 1 … 0 Properties still OK

  13. Example: a FIFO fst always outputs the first element put overrides get FIFO get fst num put err in err signals for 1 clock cycle when something goes wrong – the FIFO does not break

  14. A First Property List Safety properties • always (put=1 & num=n  next err=1) • always (put=1 & num<n  next num=num+1) • always (put=1 & num=0  next fst=in) • always (put=1 & 0<num<n  next fst=fst) • always (get=1 & put=0 & num=0  next err=1) • always (get=1 & put=0 & 0<num  next num=num-1) No restriction on the logic… n = max. FIFO size

  15. Analyzing err get 0 … put 0 … in 0 … num 0 … fst 0 … err ? … Not constrained at time 0

  16. Analyzing err: A Fix Add: • err=0 Holds at initial point in time

  17. Analyzing err get 0 0 … put 0 0 … in 0 0 … num 0 0 … fst 0 0 … err 0 ? … Nothing goes wrong…

  18. Analyzing err: A fix Add: • always (get=0 & put=0  next err=0)

  19. Analyzing err get 0 0 … put 1 0 … in 1 0 … num 0 1 … fst 0 1 … err 0 ? … Nothing goes wrong…

  20. Analyzing err: A fix • always (put=1 & num<n  next num=num+1) • always (get=1 & put=0 & 0<num  next num=num-1) Change to: • always (put=1 & num<n  next num=num+1 & next err=0) • always (get=1 & put=0 & 0<num  next num=num-1 & next err=0) Analysis does not complain about err anymore…

  21. Analyzing num get 0 … put 0 … in 0 … num ? … fst 0 … err 0 … Not constrained at time 0

  22. Analyzing num: A Fix Add: • num=0 Holds at initial point in time

  23. Analyzing num get 0 0 … put 0 0 … in 0 0 … num 0 ? … fst 0 0 … err 0 0 … Not constrained

  24. Analyzing num: A fix Add: • always (get=0 & put=0  next num=num)

  25. Analyzing num get 1 0 … put 0 0 … in 0 0 … num 0 ? … fst 0 0 … err 0 1 … An error has occurred

  26. Analyzing num: A fix • always (get=0 & put=0  next num=num) Change to: • always ((get=0 & put=0) vnext err=1  next num=num) The analysis is now happy about num… Dependencies between outputs…

  27. Analyzing fst get 0 … put 0 … in 0 … num 0 … fst ? … err 0 … Not restricted But we don’t want to specify it!

  28. Analyzing fst: A fix Logically vacuous: “fst=fst” Add: • always (num=0  free fst) A new keyword: free For the analysis: free x means that x is constrained: “don’t complain!”

  29. Analyzing fst get 0 0 1 0 … put 1 1 0 0 … in 3 5 0 0 … num 0 1 2 1 … fst 0 3 3 ? … err 0 0 0 0 … Not restricted: FIFO behavior not specified in properties

  30. Unconstrained Outputs Cases we have to be explicit about • Three causes: • It is supposed to be unspecified • We decide not to specify it (complicated) • We have forgotten to specify it Cases we want to catch

  31. Analyzing fst: A fix • always (get=1 & put=0 & 0<num  next num=num-1 & next err=0) Change to: • always (get=1 & put=0 & 0<num  next num=num-1 & next err=0 & next free fst)

  32. Analyzing fst get 0 0 … put 0 0 … in 0 0 … num 0 0 … fst 0 ? … err 0 0 … Real forgotten case

  33. Analyzing fst: A fix • always ((get=0 & put=0) v next err=1  next num=num) Change to: • always ((get=0 & put=0) vnext err=1  next num=num & next fst = fst) The analysis is now happy about fst… Luckily, we used free…

  34. New & changed properties initial values implicit behavior constant behavior • err=0 & num=0 • always (get=0 & put=0  next err=0) • always ((get=0 & put=0) vnext err=1  next num=num & next fst = fst) • always (put=1 & num<n  next num=num+1 & next err=0) • always (get=1 & put=0 & 0<num  next num=num-1 & next err=0 & next free fst)

  35. Implementation • For output s, find a trace where • Props(s) is OK • Props(s’) is OK • Exists exactly one t where s[t]≠s’[t] • free(s)[t] should be false • (this trace must be infinite)

  36. Implementation (I) Using a standard LTL model checker (NuSMV) • Find a trace satisfying: • Props(s) & Props(s’) & ◊!(s≠s’) • With the use of free: • Props(s,free_s) & Props(s’,free_s) & ◊!(~free_s & s≠s’)

  37. Implementation (II) • Property observer for safety property phi: • □OK holds iff. phi holds • Analysis for property observers: • □OK(s) & □OK(s’) ◊!(s≠s’)

  38. Discussion (I) • Forgotten case • Given an output signal s and a time t, and given the values of all other signals at all points in time, and given all values of s not at time t, do the properties force the value of s at time t? • Alternatives…

  39. Discussion (II) • Freeness • “free s” does not indicate that s can take on any value • rather, it is an artifical way of constraining s for the sake of the analysis, without actually restricting it logically

  40. Conclusion: This Analysis • Identifies forgotten cases • Which inspire forgotten properties • Forces to specify when outputs are free • Distinction between forgotten cases and underconstrainedness • Is design-independent • pre-design / multiple implementations • cost

More Related