Model Exploration with Temporal Logic Query Checking

1. Model Exploration with Temporal Logic Query Checking Arie Gurfinkel Benet Devereux Marsha Chechik Dept of Computer Science, University of Toronto

2. Model-Checking Temporal logic property Model of System (state-based) AG(p  q) AG(pAXq) Checker Engine Answer + Counter-example • Typically used for verification Correct? • Where do the properties come from? • What to do when they do not hold? So, goal is not just verification butdiscovery of propertiesthat the system should have!

3. Correctness properties: CTL CTL - Computation Tree Logic • propositional temporal logic • branching-time logic, allowing explicit quantification over possible futures • Syntax: if pand q are CTL formulae, then so are:  p, p q, p q EX p - p is true in some next states EF p - along some path, p is true in some future state; E[p U q] - along some path, pholds until q holds; EG p - along some path, p holds in every state Universal quantification: AX p , AF p, A[p U q], AG p

4. Models:Kripke Structures p  q r  p q r s1 s0 p q r s2 • Conventional state machines • M = <S, A, s0, I , R> • S is a (finite) set of states • A is a (finite) set of propositional variables • s0 is a unique initial state (s0 S) • I: S  2A is a labeling function that maps each state only the set of propositional variables that hold in it • R S  S is the (total) transition relation

5. Query Checking p  q r  p q r s1 s0 p q r s2 • Goal: speed-up design understanding • discover properties not known a priori • Temporal logic query • temporal logic formula with an “unknown” (placeholder) instead of a propositional formula • e.g. AG ?x, AG(?x  p) • solution: strongest propositional formula that makes the resulting temporal formula true • (p  q)  r is strongest solution to AG ?x • all others, e.g. p  q, implied by it • usually restrict atomic propositions • solution to AG (?x{p, q}) is p  q

6. Naïve solution • Algorithm • Take query  • Take set Aof atomic propositions of interest • |A| = n • Take all 22npropositional formulas over A • Substitute them for the placeholder • Solve resulting 22n model-checking problems • Return the strongest solution • Complexity: O(||  |S|  22n) Thus, n is vital for ensuring feasibility of query-checking!

7. Solving TLQ (Chan [CAV’00]) • Original definition • provide partial explanation when property holds • e.g. instead of AG (a  b), ask AG ?x{a, b} • answer a  b is stronger! • provide diagnostic information when property fails • e.g. if AG (req  AF ack) fails - ask AG (?x  AF ack) • Types of queries • “valid” (have a single strongest solution) • in general - $$ to determine if a query is valid • Approach • syntactic class of SMV properties with valid solutions • Implementation • on top of SMV

8. Solving TLQ (B&G [LICS’01]) p  q r  p q r s1 s0 p q r s2 • Background: queries with multiple solutions • e.g. EX ?x • solutions: p  q  r, p  q  r • incomparable! • Clearly useful in model understanding! • Types of queries: • arbitrary temporal logic formulae (not necessarily “valid”) • single “unknown” • How? • via alternating automata • Implementation • none

9. Solving TLQ (our work) • How? • Reduce to multi-valued model-checking • decision procedure that receives any De Morgan algebra, blah, blah, blah • returns witness/counter-example explaining the answer • Types of queries • arbitrary temporal logic formulae (not necessarily “valid”) • any number of placeholders • output includes “reason” why the answer is as given • Implementation • on top of our multi-valued model-checker XChek • What good is it?

10. Overview • What is query-checking? • Related work • What did we do? • Reduction to multi-valued model-checking • Advantages: • witnesses /counter-examples available • more interesting queries (arbitrary temporal logic formulae, several placeholders) • What good is it? • Case study: Cruise Control System • Queries with witnesses • Queries without witnesses • How did we do this? • Summary, conclusion, future work

11. Example: Cruise Control System (CCS) For keeping an automobile running at a certain speed • Driver accelerates to desired speed and then presses a button on steering wheel (Button = bCruise) • System maintains car speed until • The driver presses the brake pedal (Break) • The driver presses the accelerator (Accel) • The driver turns the cruise control off (Button=bOff) • The driver turns the ignition off (Ignition) • The car’s speed becomes uncontrollable (Toofast) • The system can be reactivated by presing a “resume” button (Button=bResume) • Overall size: 10 monitored, 4 controlled variables.

12. Specification of CCS @T(a) WHEN b a b a • Specified using SCR method • Output: controlled variables • Input: monitored variables • System state: modeclasses • Sets of states (modes) that partition the state space • The system is in exactly one more of each modeclass at any point • State changes – in response to events (changes in environment) • @T(a) WHEN b = a  b  a’

13. Mode TransitionTable for CCS Initial Mode: Off WHEN NOT(Ignition)

14. Event Table for Throttle Modes Events Cruise @T(Speed=slow) @T(Speed=ok) @T(Speed=fast) @F(Inmode) @T(Inmode) when(Speed=slow) @T(Inmode) when (Speed=ok) @T(Inmode) when (Speed=fast) Cruise @F(Inmode) Throttle’= tAccel tMaintain tDecel tOff Initial: Throttle=tOff

15. Applications of TLQ Solution too big! • Reachability analysis “are all modes of modeclass CC reachable?” EF(CC=Cruise) EF ?x{CC} “find all possible values of ThrottleEF ( (CC=Cruise)  ?x{Throttle}) when the system is in mode Cruise” “what modes can follow Off” EF(CC=Off EX?x{CC}) “which pairs of modes can EF(?old{CC}  EX?new{CC}) follow each other” • Invariant Discovery “find the strongest invariant” AG ?x “find invariant of mode InactiveAG ((CC=Inactive)  ?x{Ignition,Running}) w.r.t. Ignition and Running” “find invariants of all modes w.r.t AG (?x{CC}  ?y{Ignition,Running}) Ignition and Running”

16. Applications of TLQ: Guard Discovery    • a guard is a weakest propositional formula  s.t. the invariant      holds • ask a query AG (  ?pre AX (?post   )) • How does the system transition between Cruise and Inactive? AG((CC=Cruise)  ?pre{Toofast,Running,Brake} AX (?post{Toofast,Running,Brake}  (CC=Inactive))) solution: ?pre = Brake  Toofast  Running ?post = true ?pre = true ?post = Running  Toofast

17. Quick Aside: Witnesses • Example: • query: EX ?x{p} p  q r  p q r s1 s0 p q r s2 • witness: s0 • ?x = p • ?x = p s2 s1 • A witness • a trace explaining why the formula holds • produced automatically by model-checker • solutions: ?x = p ?x = p • can prefer certain witnesses over others • based on size • based on largest common prefix

18. Guided Simulation • Simulation - a means of exploring a model • provide inputs, observe outputs • Example: • goal: explore evolution of CCS into its different modes • guess (!!) which inputs result in Inactive Cruise and then CruiseOverride, etc. • slow and error prone • Guided simulation • specify objective via a query • witness serves as basis for simulation • Example: • goal: EF ?x{CC} • prefer witnesses with largest common prefix • Output: • one trace: OffInactiveCruiseOverride • sequence of events: @T(Ignition), @T(Running), @T(Button=bCruise), @T(Button=bOff) • no user input required!

19. Guided Simulation • Output: • one trace: OffInactiveCruiseOverride • sequence of events: @T(Ignition), @T(Running), @T(Button=bCruise), @T(Button=bOff) • no user input required!

20. TLQ Applications: Testing • Current approach (Gargantini, Heitmeyer [FSE’99]) • branch coverage • for each mode in mode transition table, test each even at least once • for each mode, test every no-change at least once • form CTL properties • EF((CC=Off) EX(CC=Inactive)) • EF((CC=Off) EX(CC=Off)) • witness produced by model-checker is the test case!

21. TLQ for Testing • TLQ approach: • witness to single query • EF((CC=Off) EX ?x{CC}) • coverage of the entire mode transition table • EF(?x{CC} EX ?y{CC}) • Advantages: • can minimize the number of test cases that guarantee full coverage • each solution and witness are returned as soon as they are generated! • Disadvantages: • does not cover every event (just every change and no-change) • but… can use other methods to cover events (see paper)

22. Reduction to XCTL: Example

23. Running time: • Running time of the model checker is O(|S|  ||  k) • |S| - size of state space • || - the size of the CTL formula • k - time to compute EX • Running time of query-checker (in paper) is O(|S| ||  22n k) • n - number of atomic propositions of interest • Hot off the press ([Gurfinkel’02]) • model-checker does not depend on height of the lattice representing the logic, and thus … • Running time of query-checker is O(|S|  ||  k) • (typically) slower than classical model-checker because of factor k (which depends on n)

24. Summary • Extended TLQ framework • … to allow for queries with multiple placeholders • which is useful for a variety of software engineering applications • Implemented a query-checker • using our multi-valued model-checker XChek • in addition to answers, has witnesses explaining solutions and fairness • e.g., compute invariant of CCS if Brake is pressed infinitely often • For more information: • To obtain a prototype version of XChek • send e-mail to xchek@cs.toronto.edu • Our papers: http://www.cs.toronto.edu/~chechik/publications.html

25. Future Work • Further applications of TLQ • in particular, to testing • Queries with “interesting” restrictions on placeholders • e.g. EF ?x{p, q, r}, where pand qare not true simultaneously • our framework supports it • we need a good language for expressing it • Further case studies to assess feasibility for real systems • Methodology for use of query-checking

26. Questions?Comments?Concerns?Suggestions? THANKS FOR YOUR ATTENTION!