1 / 55

Hiding the Formalism in Formal Methods

Hiding the Formalism in Formal Methods. Lori A. Clarke Laboratory for Advanced Software Engineering Research (LASER) University of Massachusetts, Amherst http://laser.cs.umass.edu/

Download Presentation

Hiding the Formalism in Formal Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Hiding the Formalism in Formal Methods Lori A. ClarkeLaboratory for Advanced Software Engineering Research (LASER) University of Massachusetts, Amhersthttp://laser.cs.umass.edu/ Work done in collaboration with George S. Avrunin, Jamieson M. Cobleigh, Rachel Cobleigh, Heather M. Conboy, Matthew B. Dwyer, Gleb Naumovich, & Leon J. Osterweil

  2. Model Checking • Includes a wide range of approaches for determining if finite models of systems are consistent with specified properties • E.g., SPIN, SMV,FLAVERS • Verifies properties about system behavior • Successfully applied to hardware, software, and various modeling languages (BPEL, Petri Nets, UML, Little-JIL) • Seeks a middle ground between testing and theorem proving • Testing requires “executable” semantics and only provides selective results • Theorem proving can deal with a wider range of properties, but usually requires more mathematical expertise

  3. Model checking is still not widely applied • State Explosion Problem • The cost of analysis can be exponential in the size of the system being analyzed • Restricted to small systems • Even with extensive optimizations, creating a concise, but valid, model requires considerable insight • Specifying properties is difficult • Notations are cumbersome • Many details must be taken into consideration

  4. Two Approaches for increasing acceptance • Automatically produce the model that is the basis for analysis • Produce a very concise, but conservative model • Incrementally add precision • Provide natural language Interfaces for creating properties

  5. Outline • FLAVERS approach for automatically creating and improving the model • Checking Properties • Improving Precision • Experimental Results • PROPEL approach for elucidating properties • Question Tree • Disciplined Natural Language • Finite-state automaton

  6. FLAVERS • FLow Analysis for VERification of Systems • Verifies properties about concurrent and sequential systems • Properties are represented as finite state automata • Checked using an efficient state propagation algorithm • Uses an abstract, event-based graph model of the system • Imprecise, but conservative • Precision can be improved incrementally

  7. Models for Concurrent Systems • One model for a concurrent system is a reachability graph • Represents all of the states a concurrent system may reach • Location within each task • Values of variables

  8. task body t1 is begin u; t2.send_synch; v; w; end t1; task t2 body is begin x; t1.rec_ synch; y; z; end t2; b,b u,b b,x u,x s,s v,s s,y w,s v,y s,z w,y v,z w,z e,e Reachability Graph

  9. Trace Flow Graph (TFG) • A TFG represents control flow through a concurrent system • Built from Control Flow Graphs for the tasks in the system • Nodes and edges are added to represent concurrency

  10. task body t1 is begin u; t2.send_synch; v; w; end t1; task t2 body is begin x; t1.rec_synch; y; z; end t2; send_synch rec_ synch TFG Construction ` ` u x synch v y w z

  11. b,b u,b u x u,x synch s,s v,s v y w,s w z w,y w,z e,e Feasible Paths b,b u,b b,x u x u,x synch s,s v,s s,y v y w,s v,y s,z w z w,y v,z w,z e,e

  12. b,b u,b u synch Infeasible Paths b,b u,b b,x u x u,x synch s,s v,s s,y v y w,s v,y s,z w z w,y v,z w,z e,e

  13. The elevator does not move while its doors are open. L(P) is the set of all strings accepted by P 1 closemove open close 2 open move closemoveopen 3 Elevator Property

  14. Control Flow Graph (CFG) • A CFG G is <N, ninitial, nfinal, E> • Associate events with nodes • G is the alphabet of G • L(G) is the language of G • The set of all strings in (G) that occur on paths from the initial node to the final node • CFG is alphabet refined • Remove nodes that do not affect the property being verified

  15. Simple Sequential Example 1: if … 1: if (stopped) then 2: open; end if; … 3: if (stopped) then 4: close; end if; … 5: move; … 2: open 3: if 4: close 5: move

  16. Proving Properties • Given a CFG G and a property P • Alphabet refine G with respect to P • Need to show L(G)  L(P) • Use data-flow analysis to propagate states of P to the nodes of G • Worst-case cost is O((NG)2  SP)

  17. 1: if 1 closemove open close 2 open 3: if move closemove open 3 State Propagation Worklist: 2, 3 , 4, 5 {1} 2: open {2} {1,2} 4: close {1} 5: move {1,3}

  18. 1: if 2: open 3: if 4: close 5: move State Propagation Worklist: 2, 3 , 4, 5 {1} 1 closemove open {2} close 2 open {1,2} move {1} closemove open 3 {1,3}

  19. 1: if 1 closemove 2: open open close 2 3: if open move 4: close closemove open 3 5: move State Propagation {1} {2} {1,2} {1} {1,3}

  20. 1: if 2: open 3: if 4: close 5: move State Propagation … 1: if (stopped) then 2: open; end if; … 3: if (stopped) then 4: close; end if; … 5: move; …

  21. Boolean Variable Constraint u S==tS=t S==fS=f S=f S==t S=t S==fS=f t f S=t S==f S==t v S==t S=tS==f S=f == is a predicate= is assignment

  22. Boolean Variable Constraint u S==tS=t S==fS=f S=f S==t S=t S==fS=f t f S=t S==f S==t v S==t S=tS==f S=f == is a predicate= is assignment

  23. Improving Precision • Use constraints to improve precision • Represented as FSAs • Given a CFG G, a property P, and constraints C1,…,Cn • Alphabet refine G with respect to (P C1 … Cn) • Want (L(G) L(C1) …L(Cn))  L(P) • Worst-case cost is O(NG2  SP  SC1 … SCn)

  24. 1: if 2: S==t 4: S==f 3: open 5: if 6: S==t 8: S==f 7: close 9: move Elevator Revisited … 1,2,4: if (stopped) then 3: open; end if; … 5,6,8: if (stopped) then 7: close; end if; … 9: move; …

  25. closemove 1 open close 1: if 2 open move closemove open 3 u 5: if S==t S==f S==t S==f t f S==f S==t v S==f S==t State Propagation Worklist: 2, 4 , 3 , 5 , 6, 8 <1,u> 2: S==t 4: S==f <1,t> <1,f> 3: open <2,t> <2,t>,<1,f> 6: S==t 8: S==f <2,t>,<1,v> 7: close 9: move

  26. closemove 1 open close 2 open move closemove open 3 u S==t S==f S==t S==f t f S==f S==t v S==f S==t State Propagation Worklist: 2, 4 , 3 , 5 , 6, 8 1: if <1,u> 2: S==t 4: S==f <1,t> <1,f> 3: open <2,t> 5: if <2,t>,<1,f> 6: S==t 8: S==f <2,t>,<1,v> 7: close 9: move

  27. closemove 1 , 7, 9 open close 2 open move closemove open 3 u S==t S==f S==t S==f t f <2,v>,<1,f> S==f S==t <1,t> v S==f S==t <1,t>,<1,f> State Propagation Worklist: 2, 4 , 3 , 5 , 6, 8 1: if <1,u> 2: S==t 4: S==f <1,t> <1,f> 3: open <2,t> 5: if <2,t>,<1,f> 6: S==t 8: S==f <2,t>,<1,v> 7: close 9: move

  28. closemove 1 open close , 7, 9 2 open move closemove open 3 u S==t S==f S==t S==f t f S==f S==t v S==f S==t State Propagation Worklist: 2, 4 , 3 , 5 , 6, 8 1: if <1,u> 2: S==t 4: S==f <1,t> <1,f> 3: open <2,t> 5: if <2,t>,<1,f> 6: S==t 8: S==f <2,t>,<1,v> <2,v>,<1,f> 7: close <1,t> 9: move <1,t>,<1,f>

  29. Automatically Add Constraints as Needed • Variable Automata - model variables that impact important predicates • Task Automata - model control flow of selective tasks

  30. b,b u,b u synch Infeasible Paths b,b u,b b,x u x u,x synch s,s v,s s,y v y w,s v,y s,z w z w,y v,z w,z e,e

  31. Experimental Results • Evaluate how FLAVERS performance scales as program size increases • Time • Memory • Number of constraints

  32. Example: Chiron • User interface system developed at UC Irvine • Uses event-based notification • Scaled by increasing the number of listened for events • Lines of code • 2 events 259 • 53 events 3,557 • Proved several properties about Chiron (Avrunin, Corbett, Dwyer, Pasareanu, Siegel) • p07 - If listener1 registers for event1 before listener2, then listener1 will be notified of event1 before listener2 • p09 - The program never terminates while a listener is listening for an event

  33. Observations about Using Constraints • In our experiments • For the vast majority of programs and properties, the constraints needed to verify a property for the smallest configuration of the system are sufficient to verify the property for larger configurations • Never needed more than 4 constraints • Have not tried to find the minimal number of constraints • Vast majority of constraints could be determined automatically using simple heuristics • A useful modeling approach • Can model aspects of the environment • Can model malicious behavior

  34. Outline • FLAVERS approach for automatically creating and improving the model • Checking Properties • Improving Precision • Experimental Results • PROPEL approach for elucidating properties • Question Tree • Disciplined Natural Language • Finite-state automaton

  35. Property Specifications • A property focuses on describing one particular aspect of system behavior • Even with such focus, it can still be difficult to write a property correctly • A property should be precise and accessible • precise enough to support unambiguous communication and automated analyses • accessible enough to be readily understood

  36. Specifications need to be… Accessible Rigorous Precise/accurate Consistent Analyzable What is the problem? • natural language is accessible “the nurse must verify the patient's identifying information before starting to transfuse a unit of blood product into the patient” • ..but it is neither rigorous nor precise • Is the nurse allowed to verify the patient's identity more than once before starting to transfuse blood? • Does the nurse have to verify the patient's identity again before transfusing the next unit of blood?

  37. What is the problem? Specifications need to be… • Accessible • Rigorous • Precise/accurate • Consistent • Analyzable • temporal logic is rigorous Linear Temporal Logic (LTL): ☐¬order_received ∨ ♢(order_received ∧ (☐¬blood_transfused ∨ (¬blood_transfused U identity_verified))) • …but not accessible to most practitioners

  38. Our Approach • Provides property templates that explicitly shows options • Extends property patterns (Dwyer, Avrunin, & Corbett 1998; 1999) • Provides multiple views of the property • Views chosen to support precision, accessibility, and user guidance • User can work with one or more of the views • Changes made in one view are reflected in the others • Formal view is rigorous enough to support verification • Implemented prototype tool, PROPEL • PROPerty Elucidation

  39. Propel Templates SCOPES BEHAVIORS Name Name Intent

  40. Question Tree View • Developed to help users select the appropriate pattern templates • One tree for scope and one for behavior • Found to also be useful for resolving detailed options

  41. EVENT: verify-patient-ID EVENT: transfuse-blood Example Property The patient’s identification must be verified prior to transfusing each unit of blood product. • Must identify events of primary interest • One or two eventsviews use the actual parameter names if they are provided

  42. After verify-patient-ID occurs, transfuse-blood is required to occur • transfuse-blood cannot occur until after verify-patient-ID has occurred Question Tree View How many events of primary interest are there? • One: event verify-patient-ID • Two: events verify-patient-ID and transfuse-blood

  43. Precedence FSA Template verify-patient-ID verify-patient-ID transfuse-blood transfuse-blood or verify-patient-ID or (verify-patient-ID, transfuse-blood) or transfuse-blood or verify-patient-ID or (verify-patient-ID, transfuse-blood) (verify-patient-ID, transfuse-blood)

  44. Precedence FSA Template verify-patient-ID verify-patient-ID transfuse-blood transfuse-blood or verify-patient-ID or (verify-patient-ID, transfuse-blood) or transfuse-blood or verify-patient-ID or (verify-patient-ID, transfuse-blood) (verify-patient-ID, transfuse-blood)

  45. Precedence DNL Template

  46. Precedence DNL Template

More Related