380 likes | 467 Views
Finite State Verification for Software Systems. Lori A. Clarke University of Massachusetts Laboratory for Advanced Software Engineering Research http://laser.cs.umass.edu/ Other contributors: G. Avrunin, J. Cobleigh, M. Dwyer, G. Naumovich, and L. Osterweil. Testing. can:
E N D
Finite State Verification for Software Systems Lori A. Clarke University of Massachusetts Laboratory for Advanced Software Engineering Research http://laser.cs.umass.edu/ Other contributors: G. Avrunin, J. Cobleigh, M. Dwyer, G. Naumovich, and L. Osterweil
Testing • can: • Uncover failures • Show specifications are not met for specific test cases • Be an indication of overall reliability • cannot: • Prove that a program will/will not behave in a particular way
Distributed Systems • Better performance, better flexibilty, but there is a cost • distributed systems are more difficult to test than sequential systems • number of execution paths can grow exponentially with the number of tasks • Testing can not even demonstrate that a system works on the selected/executed test data
Formal Verification: An Alternative to Testing • Theorem Proving • Prove properties about all possible executions • Use mathematical reasoning • Difficult and error prone • Finite State Verification • Prove properties about all possible executions • Reason about a finite abstract, model of the system • not as powerful as theorem proving • Almost a totally automated process
Finite State Verification: State of the Art • Reachability Based Approaches • Symbolic Model Checking -- SPIN or SMV • Exponential worst-case bound • Integer Linear Constraints • INCA • Exponential worst-case bound • Data Flow Analysis • FLAVERS • Polynomial worst-case bound
Ada, Java, C++, Jovial Architecture of FLAVERS Property Property Translator Event alphabet FSA Consistent TFG System Translator ReasoningEngine System Inconsistent+ counter example
FLAVERS: Where is the magic? • System Model • Primarily event-based, not state-based • Reason about sequences of events • Does not enumerate all reachable states • Includes all “relevant” executions • Usually over-approximates relevant executable behaviors • Model is conservative wrt a property • Incrementally refine model as needed • add state information that is demonstrated to be needed
start 1,6 <0,0> 1,6 <1,0> 1,6 <0,1> 1,6 <1,1> 2 7 ... ... ... ... 2,6 <1,0> 2,6 <0,1> 2,6 <1,1> 2,6 <0,1> 5 Enumerating the State Space T1 T2 6 1 8 3 4 9
2 7 5 Enumerating the event space 1,6 e1 T1 T2 6 1 2,6 1,7 e1 e2 e1 3,6 2,7 1,8 e1 e2 8 e2 3 3,7 2,8 4 e2 3,8 e3 9 4 e3 5,9
T1 T2 e1 6 1 2 7 e2 8 3 4 5 5 e3 9 FLAVERS model of the system T1 T2 e1 1 e2 8 4 e3 9
Modeling the System: FLAVERS • Automatically creates the program model from the source code • Instead of the state space, explicitly represents interleaved execution via edges • Compute MHP • Optimize the representation • Model only events of interest • Conservative with respect to a property
Some comments on the model • Works well if the events of interest are relatively sparse in the overall system • E.g., method invocations, task interaction • Inter-procedural analysis done via in-lining • An imprecise, but conservative, representation • Too imprecise for deadlock • Seems to greatly reduce the analysis cost...
Ada, Java, C++, Jovial Architecture of FLAVERS Property Property Translator Event alphabet FSA Consistent TFG System Translator ReasoningEngine System Inconsistent+ counter example
The elevator always closes its doors before moving close move 0 open close open 1 move 2 close, open,move Example Property FSA
Reasoning engine: State Propagation • States of the property are propagated through the TFG • The property is proved if only accepting (non-accepting) states are contained in the final node of the TFG • overall complexity is O(N2S) • N is the size of the model of the system • S is the number of states • guaranteed to terminate
Elevator Controller 1: if void main() { … 1,2,4: if (elevatorStopped) {... 3: openDoors(); } ... 5,6,8: if (elevatorStopped) {... 7: closeDoors(); } 9: moveToNextFloor(); } 3: open 5: if 7: close 9: move
Property close move 0 open close open 1 1: if move close open 2 move 3: open 5: if 7: close 9: move Elevator Controller <0> Worklist: 3, 5, 7,9 <1> <0,1> <0> <0,2>
Property close move 0 open close open 1 move close open 2 move Elevator Controller <0> Worklist: 3, 5, 7,9 1: if void main() { … 1,2,4: if (elevatorStopped) {... 3: openDoors(); } ... 5,6,8: if (elevatorStopped) {... 7: closeDoors(); } 9: moveToNextFloor(); } 3: open <1> 5: if <0,1> 7: close <0> 9: move <0,2>
Conservative Analysis • if a property is found to be valid, it is definitely true • if a property is found to be invalid • An invalid trace may correspond to an error • The invalid traces do not correspond to executable behavior
Constraints Ada, Java, C++, Jovial Revised Architecture of FLAVERS Property Property/Constraint Translator Event alphabet FSA Consistent TFG System Translator ReasoningEngine System Inconsistent+ counter example
Elevator Controller 1: if void main() { … 1,2,4: if (elevatorStopped) {... 3: openDoors(); } ... 5,6,8: if (elevatorStopped) {... 7: closeDoors(); } 9: moveToNextFloor(); } 2: S==true 4: S==false 3: open 5: if 6: S==true 8: S==false 7: close 9: move
Example constraint for a boolean variable unknown S==falseS=false S==trueS=true S=false S==true S=true S==falseS=false true false S=true S==true S==false viol == is a predicate = is assignment S==trueS=trueS==falseS=false
Example constraint for a boolean variable unknown S==falseS=false S==trueS=true S=false S==true S=true S==falseS=false true false S=true S==true S==false viol == is a predicate = is assignment S==trueS=trueS==falseS=false
Property close move 0 open close open 1 move close open 2 move Constraint 0 S==true S==false S==true S==false 1 2 S==false S==true viol S==true S==false State Propagation Worklist: 2, 4, 3, 5, 6 ,8, 1: if <0,0> 2: S==true 4: S==false <0,2> <0,1> 3: open <1,1> 5: if <1,1>,<0,2> 6: S==true 8: S==false <1,1>,<0,v> 7: close 9: move
Property close move 0 open close open 1 move close open 2 move Constraint 0 S==true S==false S==true S==false 1 2 S==false S==true viol S==true S==false State Propagation Worklist: 2, 4, 3, 5, 6 ,8, 7,9 1: if <0,0> 2: S==true 4: S==false <0,2> <0,1> 3: open <1,1> 5: if <1,1>,<0,2> 6: S==true 8: S==false <1,1>,<0,v> 7: close 9: move
Property close move 0 open Worklist: 2, 4, 3, 5, 6, 8, 7, 9 close open 1 move close open 2 move Constraint 0 S==true S==false S==true S==false 1 2 <1,v>,<0,2> S==false S==true <0,1> viol Worklist: 2, 4, 3, 5, 6, 8, 7 S==true S==false <0,1>,<0,2> State Propagation 1: if <0,0> 2: S==true 4: S==false <0,2> <0,1> 3: open <1,1> 5: if <1,1>,<0,2> 6: S==true 8: S==false <1,1>,<0,v> 7: close 9: move
Constraints • Used to add semantic information • Can model many different kinds of information • Variables or flow of control • Missing components • Environment • Users • Can provide automated assistance
Evaluating FLAVERS • Experimental Evaluation • Case studies: • Communication Protocols: 3-way handshake and alternating bit • demonstrated the importance of modeling the environment • Architectural Design written in Wright • demonstrated importance of doing analysis early • Industrial demonstrations: • SAIC: ADS Command Instruction Sets, HLA • Northrop: B2 Jovial systems • MCC: Quest project, C++ systems
Need a paradigm shift! • Software systems are increasingly used in critical applications • Medicine • Transportation • Communication • Finance • Cannot continue to do business as usual • Testing as the prime validation technique is costly and ineffective
New life to an old vision • Significant validation throughout the software lifecycle: • architectural design • low level design • coding • debugging • Maintenance • Finite state verification is an extremely general approach
Early fault detection • Requirement specs ==> properties • Design specs ==> properties • properties X system representations ==> early feedback
Future Plans • Difficult to predict performance • For any FSV technique • For FLAVERS: Adding constraints increases the worst-case bound, but may (or may not) improve performance • Need more automated support • Analysis to help select additional information to be modeled • Abstraction techniques for modeling
Future Work • Explore alternative state propagation algorithms • E.g., Initial verification vs re-verification • Improve counter-example generation • Short paths • User guidance • Better support for specifying properties • Build on property pattern specifications • FSA templates combined with Disciplined Natural Language
action response or (action,response) This property is repeatable. This property is not repeatable. repetition phrase Response Property FSA template: action action response response or (action,response) response (action,response) DNL template: Multiple occurrences of action eventually result in only one occurrence of response. Action may occur zero times. Action may occur zero times. The Repetition phrase describes whether or not the property is repeatable. This property is repeatable. Response cannot occur before the first action occurs. repetition phrase
action Instantiated Response Property completed FSA Competed FSA template… action action response response or (action,response) response response (action,response) …and its DNL representation of your property. …and a DNL template. Multiple occurrences of action eventually result in only one occurrence of response. Action may occur zero times. Response cannot occur before the first action occurs. This property is repeatable. Multiple occurrences of action eventually result in only one occurrence of response. Action may occur zero times. Response cannot occur before the first action occurs. This property is repeatable. This property is repeatable.
Future Work • Software composition techniques • Contractual composition • Integration with testing • Better approaches for dealing with dynamism • Full lifecycle support • Support for Java • Experimental Evaluation
Conclusions • FLAVERS is effective at verifying a wide range of interesting properties • Very general approach that can be applied to any event flow model • Much easier to use than theorem proving based techniques • Can be mostly automated • Hides the “formal” in formal methods • Is it easy enough?