700 likes | 784 Views
Automatic Generation of Path Conditions for Concurrent Timed Systems. Doron Peled, University of Warwick Joint work with Saddek Bensalem, Stavros Tripakis, Hongyang Qu. Unit testing: Selection of test cases (for white-box testing). The main problem is to select a good coverage
E N D
Automatic Generation of Path Conditions for Concurrent Timed Systems Doron Peled, University of Warwick Joint work with Saddek Bensalem, Stavros Tripakis, Hongyang Qu
Unit testing: Selection of test cases (for white-box testing) The main problem is to select a good coverage criterion. Some standard options are: • Cover all paths of the program. • Execute every statement at least once. • Each decision (diamond node on flow chart) has a true or false value at least once. • Each condition predicate is taking each truth value at least once. • Check all possible combinations of conditions in each decision.
Cover all the paths of the program (skip) Infeasible. Consider the flow diagram on the left. It corresponds to a loop. The loop body has 5 paths. If the loops executes 20 times there are 5^20 different paths! May also be unbounded: number of iterations depend on data. So also forget about covering all executions!
How to cover the executions? if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; • Choose values for A,B,X at the beginning that would force the right path/conditions/predicates. • Value of X may change, depending on A,B. • What do we want to cover? Paths? Statements? Conditions?
By choosing A=2,B=0,X=3 each statement will be chosen. The case where the tests fail is not checked! if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; Statement coverageExecute every statement at least once Now x=1.5
Can be achieved using A=3,B=0,X=3 A=2,B=1,X=1 Problem: Does not test individual predicates. E.g., when X>1 is erroneous in second decision. if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; Decision coverageEach decision (diamond node in flow graph) tested with true and false outcome at least once.
For example: A=1,B=0,X=3 A=2,B=1,X=0 lets each condition be true and false once. Problem:covers only the path where the first test fails and the second succeeds. if (A>1)&(B=0) then X=X/A; if (A=2)|(X>1) then X=X+1; Condition coverage (skip)Each predicate has a trueand false value at least once.
Preliminary:Relativizing assertions A (B) : x1= y1 * x2 + y2 /\ y2 >= 0 Relativize B) w.r.t. the assignment becomes B) [Y\g(X,Y)] (I.e., (B) expressed w.r.t. variables at A.) (B)A =x1=0 * x2 + x1 /\ x1>=0 Think about two sets of variables,before={x, y, z, …} after={x’,y’,z’…}. Rewrite (B) using after, and the assignment as a relation between the set of variables. Then eliminate after. Here: x1’=y1’ * x2’ + y2’ /\ y2’>=0 /\x1=x1’ /\ x2=x2’ /\ y1’=0 /\ y2’=x1now eliminate x1’, x2’, y1’, y2’. Y=g(X,Y) (y1,y2)=(0,x1) B A (y1,y2)=(0,x1) B
Verification conditions: tests B T F C) is transformed toB)= t(X,Y) /\C) D) is transformed toB)=t(X,Y) /\ D) B)= D) /\ y2x2 t(X,Y) C D B T F y2>=x2 C D
How to find values for coverage? • Put true at end of path. • Propagate path backwards. • On assignment, relativize expression. • On “yes” edge of decision node, add decision as conjunction. • On “no” edge, add negation of decision as conjunction. • Can be more specific when calculating condition with multiple condition coverage. A>1/\B=0 no yes X=X/A A=2\/X>1 true no yes X=X+1 true
How to find values for coverage? (A2 /\ X/A>1) /\ (A>1 & B=0) A>1/\B=0 A2 /\X/A>1 no yes Need to find a satisfying assignment: A=3, X=6, B=0 Can also calculate path condition forwards. X=X/A A 2/\X>1 A=2\/X>1 true no yes X=X+1 true
How to cover a flow chart?(skip) • Cover all nodes, e.g., using search strategies: DFS, BFS. • Cover all paths (usually impractical). • Cover each adjacent sequence of N nodes. • Probabilistic testing. Using random number generator simulation. Based on typical use. • Chinese Postman: minimize edge traversalFind minimal number of times time to travel each edge using linear programming or dataflow algorithms.Duplicate edges and find an Euler path.
Test cases based on data-flow analysis (skip) • Partition the program into pieces of code with a single entry/exit point. • For each piece find which variables are set/used/tested. • Various covering criteria: • from each set to each use/test • From each set to some use/test. X:=3 t>y x>y z:=z+x
Some real life story • An expert programmer inspects the code of NASA MER. • He observe using his experience and intuition that some execution path is suspicious. • He decides how to force this path to execute, e.g., by figuring some inputs and initial values. • He executes the path, showing his supervisor the presence of an error. • We want to build some tools to help him with this process. • We’ll use LTL to help with formalizing the intuition on where the error is.
Learning from another technique: Model Checking • Automaton description of a system B. • LTL formula . Translate into an automaton P. • Check whether L(B) L(P)=. • If so, S satisfies . Otherwise, the intersection includes a counterexample. • Repeat for different properties. ¬
Unit Checking Unit Testing Model Checking
LTLAut Model Checker Path Path condition calculation Flowchart Compiler Transitions First order instantiator Test monitoring New: Test case generation based on LTL specification
Path conditions • Path in flow chart multiple executions following path. • First order formula. • All executions of a path must start with initial values satisfying the path condition. • In deterministic code, there can be only one execution starting with particular values, hence all executions starting with initial values satisfying the path condition will follow that path. • In nondeterministic code, each such initial value has an execution following a path. May need to insert synchronizing code. • Generalizations: include inputs, being more specific about decisions made in path.
Goals • Verification of software. • Compositional verification. Use only a unit of code instead of the whole code. • Parameterized verification. Verifies a procedure with any value of parameters in “one shot” • Generating test cases via path conditions: A truth assignment satisfying the path condition. Helps derive the demonstration of errors. • Generating appropriate values to missing parameters.
¬at l2 at l2/\ xy ¬at l2 at l2/\ x2y Spec: ¬at l2U (at l2/\ xy /\ (¬at l2/\(¬at l2U at l2 /\ x2y ))) Observation:each node hasconjunctions of predicates onprogram variables and programcounters • Automatic translation of LTL formula into an automaton [GPVW95] • LTL is interpreted over finite sequences. • Can use other (linear) specification. • Property specifies the path we want to find (SPIN: never claim),not the property that must hold for all paths (for this, take the negation).
Divide and Conquer • Intersect property automaton with theflow chart, regardless of the statements and program variables expressions. • Add assertions from the property automaton to further restrict the path condition. • Calculate path conditions for sequences found in the intersection. • Calculate path conditions on-the-fly. Backtrack when condition is false.Thus, advantage to forward calculation of path conditions (incrementally).
l2:x:=x+z l3:x<t l1:… Spec: (only program counters here)¬at l2U (at l2/\¬at l2/\(¬at l2U at l2)) at l2 l2:x:=x+z ¬at l2 X = ¬at l2 at l2 l3:x<t ¬at l2 at l2 l2:x:=x+z Either allexecutions of a path satisfy the formula or none. at l2 Sifts away path not satisfying formula. Then calculate path condition.
l2:x:=x+z l3:x<t l1:… Spec: ¬at l2U (at l2/\ xy /\ (¬at l2/\(¬at l2U at l2 /\ x2y ))) xy l2:x:=x+z ¬at l2 X = at l2/\ xy l3:x<t x2y ¬at l2 l2:x:=x+z Only some executions of path may satisfy formula at l2/\ x2y Modify calculation of path condition to incorporate property
Calculating the intersection of the property automaton and flow graph (abstract variables away). ¬a a a <>a s1 s2 q1 s3 q2 ¬a a Acceptance isdetermined bypropertyautomaton. s1,q1 s2,q1 s1,q2 s3,q2
How to generate test cases • Take the intersection of an LTL automaton (for a never claim) with the flow graph. Some paths would be eliminated for not satisfying the assertions on the program counters. • Seeing same flow chart node does not mean a loop: program variables may value. Use iterative deepening. • For each initial path calculate the path condition. Backtrack if condition simplifies to false. • Report path condition based on flow graph path+LTL assertions. • Always simplify conditions!
How the LTL formula directs the search Consider (x=4)U (x=5/\o…) x=4 x<5 false true x=5 y:=7 x:=x+1
How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 x<5 false true x=5 y:=7 x:=x+1
How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=4 x<5 false true x=5 y:=7 x:=x+1
How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=4 x<5 false true x=5 X=4 y:=7 x:=x+1
How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=4 x<5 false true x=5 X=4 This is in acontradiction y:=7 x:=x+1 X=4 x<5 true
How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=5 x<5 false true x=5 X=4 y:=7 x:=x+1
How the LTL formula directs the search Consider x=4U (x=5/\o…) x=4 X=5 x<5 false true x=5 X=4 y:=7 x:=x+1
l0 Example: GCD l1:x:=a l2:y:=b l3:z:=x rem y l4:x:=y l5:y:=z no l6:z=0? yes l7
l0 Example: GCD l1:x:=a l2:y:=b Oops…with an error (l4 and l5 were switched). l3:z:=x rem y l4:y:=z l5:x:=y no l6:z=0? yes l7
Why use Temporal specification • Temporal specification for sequential software? • Deadlock? Liveness? – No! • Captures the tester’s intuition about the location of an error:“I think a problem may occur when the program runs through the main while loop twice, then the if condition holds, while t>17.”
l0 Example: GCD l1:x:=a l2:y:=b a>0/\b>0/\at l0 /\at l7 l3:z:=x rem y at l0/\a>0/\b>0 l4:y:=z l5:x:=y no l6:z=0? yes at l7 l7
l0 Example: GCD l1:x:=a l2:y:=b a>0/\b>0/\at l0/\at l7 l3:z:=x rem y Path 1: l0l1l2l3l4l5l6l7a>0/\b>0/\a rem b=0 Path 2: l0l1l2l3l4l5l6l3l4l5l6l7a>0/\b>0/\a rem b0 l4:y:=z l5:x:=y no l6:z=0? yes l7
Potential explosion Bad point: potential explosion Good point: may be chopped on-the-fly
Now we add time • Detailed model, for each transition we have 4 parameters [l, u, L, U]: • l Needs to be enabled at least that much. • u Cannot be enabled without taken longer than that. • L Least time for transformation to occur (after been chosen). • U Transformation cannot take more than that.
Translation to timed automata s1at l c1c2x2:=0 c1c2x2:=0 c1c2x1:=0 c1c2x1:=0 c1c2x1,x2:=0 c1c2 c1c2 c1c2 s3,at lx2<u2x1<u1 c1c2 c1c2 s2,at lx1<u1 s4,at lx2<u2 c1c2x2:=0 c1c2x1:=0 Timing out the enabledness:Zero counters,Cannot wait enabled too much.
Translation to timed automata s1at l c1c2x2:=0 c1c2x2:=0 c1c2x1:=0 c1c2x1:=0 c1c2x1,x2:=0 c1c2 c1c2 c1c2 s3,at lx2<u2x1<u1 c1c2 c1c2 s2,at lx1<u1 s4,at lx2<u2 c1c2x2:=0 c1c2x1:=0 ac x1l1x1:=0 bc x1l1x1:=0 x2l2x2:=0 x2l2x2:=0 bc ac s5x1<U1 s6x2<U2 Can fire only if waited enough,Zero counters again.
Translation to timed automata s1at l c1c2x2:=0 c1c2x2:=0 c1c2x1:=0 c1c2x1:=0 c1c2x1,x2:=0 c1c2 c1c2 c1c2 s3,at lx2<u2x1<u1 c1c2 c1c2 s2,at lx1<u1 s4,at lx2<u2 c1c2x2:=0 c1c2x1:=0 ac x1l1x1:=0 bc x1l1x1:=0 x2l2x2:=0 x2l2x2:=0 bc ac s5x1<U1 s6x2<U2 x1L1 x2L2 af bf s8 s7
Its easy to select an interleaved sequence. But due to time limitations, it may execute in a different order. Just the order on events from the same process and using same variables is to be considered. Should we really look at paths? a a b b c d c Samevariable d Sameprocess
Generate an automaton for all consistent interleavings b a a b c d a b c Intersect this automaton with automaton for system.Calculate “partial order” condition: start from leaves.When there is a choice, usedisjunct. d c b d c
Generate an automaton for all consistent interleavings b a a b c d c b d c
Generate an automaton for all consistent interleavings b a a b c d c b d c
Generate an automaton for all consistent interleavings b a a b c d c b d c