1 / 80

Lori A. Clarke 教授

Lori A. Clarke 教授. 麻萨诸塞大学(阿姆赫斯特)计算机科学系 先进软件工程研究实验室 University of Massachusetts, Amherst. Laboratory for Advanced Software Engineering Research Clarke@cs.umass.edu http://laser.cs.umass.edu/. Finite State Verification:.

clark-benjamin
Download Presentation

Lori A. Clarke 教授

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lori A. Clarke教授 麻萨诸塞大学(阿姆赫斯特)计算机科学系 先进软件工程研究实验室 University of Massachusetts, Amherst. Laboratory for Advanced Software Engineering Research Clarke@cs.umass.eduhttp://laser.cs.umass.edu/

  2. Finite State Verification: • An Emerging Technology for Validating Software Systems • 一种介于测试和验证之间的技术和方法 • 支持这种方法的自动工具。

  3. Sorry State of Affairs • Testing consumes about half the cost of s/w development • Maintenance consumes about 80% of the full life cycle costs--much of that devoted to testing • Most companies use ad hoc QA practices • Unhappy with the results; Unhappy with the cost • Failed projects • Delayed product releases

  4. Testing • can: • Uncover failures • Show specifications are (not) met for specific test cases • Be an indication of overall reliability • cannot: • Prove that a program will/will not behave in a particular way

  5. Must do better! • Increasing number of high assurance applications • Medical applications • Flight control software • Electronic commerce • Increasing number of complex systems • Systems of systems • Distributed systems

  6. Distributed Systems • Better performance, better flexibility, but there is a cost • distributed systems are more difficult to test than sequential systems • number of execution paths can grow exponentially with the number of processes • Testing can not even demonstrate that a system works on the selected/executed test data

  7. 1,6 2,6 1,7 2 3,6 2,7 1,8 3,7 2,8 5 3,8 4 5,9 Complexity of Distributed Systems T1 T2 6 1 7 8 3 4 9

  8. 1,6 2,6 1,7 2 3,6 2,7 1,8 3,7 2,8 5 3,8 4 5,9 Uncertainty of Testing T1 T2 6 1 7 X: =2 8 3 X:=1 4 9 X==?

  9. Formal Verification: An Alternative to Testing • Theorem Proving Based Verification • Use mathematical reasoning • Prove properties about all possible executions • Difficult and error prone • Finite State Verification • Reason about a finite model of the system • Prove properties about all possible executions, but not as powerful as theorem proving • Almost a totally automated process

  10. Spectrum of Difficulty Ad-hoc Testing SystematicTesting Finite State Verification Theorem Proving • Arbitrary testcases • Reqts based test planning • Requirements captured as properties • Properties guaranteed on all possible executions

  11. Finite State Verification (FSV) • Holds the promise of providing a cost effective way of verifying important properties about a system • Not all faults are created equal • Invest effort into most important properties • Several promising prototypes • Reachability Based Model Checking • SPIN or Symbolic Model Checking (SMV) • Flow Equations • Integer Necessary Conditions (INCA) • Data Flow Analysis • FLAVERS

  12. High-Level Architecture of FSV Systems Property Property Translator Property Representation System System Model System Translator Property Verified ReasoningEngine Counter Examples for Model

  13. Conservative Analysis （保守型、守性型分析） • If property verified, property holds for all possible executions of the system • If property not verified: • An errorOR • A spurious result（虚假的） • System model abstracts information to be tractable • Conservative abstractions over-approximate behavior • If inconsistency relies upon over-approximations, then a spurious result • e.g. counter example corresponds to an infeasible path

  14. System Model • Depends on property being verified • Eliminate information that does not impact the proof • Abstraction techniques allows “states” in the model to be reduced/collapsed

  15. Some Properties of Properties • State-based versus event-based • Once temperature is greater than 100 degrees, lock is true • Elevator door closes before elevator moves • Single locations versus (sub)paths • Deadlock or race conditions • Sequences of states or events • Safety versus Liveness

  16. A quick look at three approaches to FSV • Model Checking • Flow Equations • Data Flow Analysis

  17. 1, Model Checking • Properties usually expressed in a temporal logic • System represented as a (possibly “abstracted”) reachability graph • State based • Reasoning engine propagates valid subformulas through the graph

  18. High-Level Architecture of Model Checking Temporal Logic Property Property Translator Property Representation State-based Reachability Graph System System Translator Property Verified Subformula propagation Counter Examples for Model

  19. Representing Properties • CTL operators • G - globally • F - future • X- next • U - until • At a state in the model: A- , E- , • AG p means that for all paths from this state, p is true and will remain true • AF p means that for all paths from this state, p will eventually be true • EG p means that for some path from this state, p is true and will remain true • EF p means that for some path from this state, p will eventually be true

  20. AF p AF p Propagating Propositions p AF p AF p

  21. AF p Propagating Propositions X p p EF p

  22. Propagation rules • For each formula type • Only need to look at a node’s successors • Need a reachability graph that shows the states (i.e., the values) of the variables • Example: process 1 can be null, trying to obtain the lock, or in its critical region (n1, t1, c1). process 2 can be null, trying to obtain the lock, or in its critical region (n2, t2, c2)turn is a variable that indicates which process can obtain the lock ( 0,1,2)

  23. Example: mutual exclusion protocol* reachability graph n1,n2,turn=0 n1,t2,turn=2 t1,n2,turn=1 n1,c2,turn=2 t1,t2,turn=2 c1,n2,turn=1 t1,t2,turn=1 c1,t2,turn=1 t1,c2,turn=2 (process1,process2,turn) *McMillan

  24. Example Property • Property: AG(t1=>AF c1) • If process1 tries to get the lock (t1) then eventually it gets into its critical region (c1) • Note, would like to prove this for all processes but FSV approaches usually must instantiate property (and system)

  25. AF c1 AF c1 AF c1 AF c1 Example: propagation AG(t1=>AF c1) n1,n2,turn=0 AF c1 n1,t2,turn=2 t1,n2,turn=1 AF c1 n1,c2,turn=2 t1,t2,turn=2 c1,n2,turn=1 AF c1 t1,t2,turn=1 c1,t2,turn=1 t1,c2,turn=2 AF c1 AF c1

  26. Need to continue propagating • ( t1=>AF c1 ) means ( AF c1  ¬ t1 )

  27. Example: propagation AG(t1=>AF c1) t1=> AF c1 n1,n2,0 t1=> AF c1 t1=> AF c1 n1,t2,2 t1,n2,1 t1=> AF c1 t1=> AFc1 t1=>AF c1 n1,c2,2 t1,t2,2 c1,n2,1 t1,t2,1 t1=> AF c1 c1,t2,1 t1,c2,2 t1=> AF c1 t1=> AF c1

  28. Example: propagation AG(t1=>AF c1) Connected region where all nodes have t1=> AF c1 ==> AG( t1=> AF c1) t1=> AF c1 n1,n2,0 t1=> AF c1 t1=> AF c1 n1,t2,2 t1,n2,1 t1=> AF c1 t1=> AFc1 t1=>AF c1 n1,c2,2 t1,t2,2 c1,n2,1 t1,t2,1 t1=> AF c1 c1,t2,1 t1,c2,2 t1=> AF c1 t1=> AF c1

  29. Example: propagation AG(t1=>AF c1) AG(t1=>AF c1) n1,n2,0 AG(t1=>AF c1) AG(t1=>AF c1) n1,t2,2 t1,n2,1 AG(t1=>AF c1) n1,c2,2 t1,t2,2 c1,n2,1 t1,t2,1 AG(t1=>AF c1) AG(t1=>AF c1) AG(t1=>AF c1) c1,t2,1 t1,c2,2 AG(t1=>AF c1) AG(t1=>AF c1)

  30. Formula Propagation • Propagate until no change • propagate from smaller to larger subformulas • keep all the formulas associated with a nodeAF c1, t1=>AF c1 ,AG(t1=>AF c1) • “smart” algorithm: linear in the size of model and size of the formula • But, model is exponential • Many optimization techniques • Symbolic model checking n1,t2,2

  31. a 0 1 b b 0 0 1 1 a 0 1 c c c c c 0 0 0 0 1 1 1 1 0 1 b b 0 0 0 0 1 1 1 1 1 1 0 1 1 0 c 0 1 1 1 Symbolic Model Checking • With abstraction, nodes may represent sets of values • Ordered Binary Decision Diagram (OBDD) ab+c Note: order is a, b, c

  32. a a 0 1 0 1 b b b 0 1 0 0 1 1 1 a 1 0 1 c c c c c c c 1 1 0 0 0 0 0 0 1 1 1 1 1 1 0 1 b b 0 0 0 0 0 0 1 1 1 1 1 1 1 1 0 1 1 a 0 0 1 c b 0 0 1 c 1 0 Symbolic Model Checking ab+c 1 1

  33. Order Binary Decision Diagram • Order of variables can effect the size of the model • In general, can not determine the optimal order without trying them all

  34. x1 x1 0 1 0 y1 y1 0 1 x2 0 1 1 1 0 x2 x2 y1 0 0 1 0 1 y2 y2 y2 1 0 1 0 1 0 0 1 0 1 1 Example of OBDDs (x1 v x2)  (y1 v y2)

  35. Some observations: Model Checking • Worst case bound linear in size of the model • Model exponential • Experimentally often very effective • Not clear if model checking or symbolic model checking is superior • Depends on the problem

  36. 2, Flow Equations • Model system as finite state automata • Use extended network flow inequalities to capture legal flow through a concurrent system • Represent negation of the property as a set of inequalities

  37. System Property System Translator Property Translator Set ofInequalities FSA’s Set of Inequalities Property Verified(no solution) FSA Translator Integer Linear Programming System Counter Examples for Model (solution) High-Level Architecture of INCA IntegerNecessaryConditionAnalyzer

  38. x10 = x9 + x12x11 = x10x11 = x12 + x13x9 = 1x13=1 x1 = x0 + x3x2 = x1x2 = x3 + x4x0 = 1x4 = 1 x6 = x5 + x7x6 = x7 + x8x5 = 1x8 = 1 Example: Task Flow Equations x0 x9 x5 x1 x10 a’ x3 x6 x12 b a x7 x11 x2 b’ x8 x13 x4 Flow in to a node = Flow out of a node

  39. Example: Inter-task Flow Equations x0 x9 x5 x1 x10 x3 x12 a’ b a x11 x2 x6 x7 b’ x13 x4 x8 x2 = x6x11 = x7 + x8 Rendezvous are always matched:# calls = # accepts

  40. Example: Require Non-Negative Flow x0 x9 x5 x1 x10 x3 x12 a’ b a x11 x2 x6 x7 b’ x13 x4 x8 j: 0  xj Flow over edges is non-negative

  41. Example: Property x0 x9 x5 x1 x10 x3 x12 a’ b a x11 x2 x6 x7 b’ x13 x4 x8 Are there more occurrences of event a then event b? Property: For all paths, event a occurs more than event b Property Equation: x2 > x11 Property Complement: x2  x11

  42. Solving the Set of Inequalities • Determine if combined system of inequalities is consistent • Use integer linear programming • If consistent, there is a set of flows through automata that violate the property • Provides guidance for trace through the model (but may not be executable)

  43. Solving for a property x1 = 1 + x3x2 = x1x2 = x3 + 1x6 = 1 + x7x6 = x7 + 1x10 = 1 + x12x11 = x10x11 = x12 + 1 x2 = x6x11 = x7 + x8 j: 0  xj x2  x11 Task Flow Equations Inter-Task Flow Equations Non-Negative Flow Property Complement Does this set of inequalities have a solution?

  44. Solving for a property x1 = 1 + x3x2 = x1x2 = x3 + 1x6 = 1 + x7x6 = x7 + 1x10 = 1 + x12x11 = x10x11 = x12 + 1 x2 = x6x11 = x7 + x8 j: 0  xj x2  x11 Solution exists e.g., x3, x7, x12=0, all other xi=1 => property does not hold

  45. Counter example x0 x9 x5 x1 x10 x3 x12 a’ b a x11 x2 x6 x7 b’ x13 x4 x8 Solution exists e.g., x3, x7, x12=0, all other xi=1 => property does not hold

  46. Some Limitations • Integer Linear Programming has an exponential worst case bound • Inter-process order information is not preserved • only checks whether event counts are consistent • Like most static techniques, may produce spurious results

  47. Some Benefits • Does not enumerate the state space! • Integer linear Programming is often very efficient • Empirical evidence: linear inequality systems usually grow linearly and take sub-exponential times to solve • In practice, INCA is often an effective technique

  48. 3, Data Flow Analysis: FLAVERS FLow Analysis for VERification of Systems • Represents property as a finite state automaton • System model is collection of annotated control flow graphs • Inter-process communication and interleavings are represented with additional edges • does not enumerate all reachable states • over-approximates relevant executable behaviors • Reasoning engine based on data flow analysis

  49. Property Property Translator FSA Collection of annotated CFG’s System System Translator Property Verified State Propagation (ControlFlow Graphs) Counter Examples from Model High-Level Architecture of Data Flow Analysis

  50. 2 7 5 Modeling the System 1,6 T1 T2 6 1 2,6 1,7 3,6 2,7 1,8 8 3 3,7 2,8 4 3,8 9 State explosion 4 5,9

More Related