1 / 42

Formal Methods Quick Tutorial

Formal Methods Quick Tutorial. Ricky W. Butler Oct 22, 2003. http://shemesh.larc.nasa.gov/fm. Outline. Motivation (3 minutes) Formal Methods by way of example Theorem Proving (12 minutes)

kasen
Download Presentation

Formal Methods Quick Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Formal Methods Quick Tutorial Ricky W. Butler Oct 22, 2003 http://shemesh.larc.nasa.gov/fm

  2. Outline • Motivation (3 minutes) • Formal Methods by way of example • Theorem Proving (12 minutes) • Model Checking state machines (12 minutes) • A Few More Thoughts (3 minutes)

  3. “There are no stupid questions”

  4. Motivation Say, I think I see where we went off. Isn’t eight times seven fifty-six?

  5. Software Underpins Everything In Aviation • Increased Capabilities • Almost any breakthrough in any field will depend on advanced computer systems • And software is where most of the complexity goes • Software Costs Dominate • Development costs (More than half of the non-recurring development of the Boeing 777) • Integration (only way of testing it is all-up simulation) • Certification (based on process, not product) • Major Safety Concerns • Reliability (dealing with hardware failures) • Correctness (no faults in its own design) • Man-machine interaction (issue in majority of recent crashes) • Certification (complexity forces FAA to rely on DERs)

  6. Example System f(x,B) x System g(x) B h(B) x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF safe_prop: THEOREM f(x,B) >= g(x) * h(B)

  7. Design Verification In the beginning, there was Simulation: • Build a model • Feed IT inputs • Analyze results

  8. Verification by Simulation x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF safe_prop: THEOREM f(x,B) >= g(x) * h(B) xBfghsafe_prop ----------- ---------------------------- -------- 0 T 0 0 60 TRUE 10 F 1200 20 0 TRUE 50 T 2500 0 60 TRUE 30 F 3600 60 0 TRUE 30 T 2700 60 60 FALSE

  9. Formal Verification x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF safe_prop: THEOREM f(x,B) >= g(x) * h(B) IF x > 30 AND B THEN x*x ELSE 90*x ENDIF >= g(x) * h(B) B NOT B IF x > 30 THEN x*x ELSE 90*x ENDIF >= g(x) * h(B) 90*x >= g(x) * h(B) 90*x >= g(x) * 0 90*x >=0

  10. Formal Verification (cont.) x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF B IF x > 30 THEN x*x ELSE 90*x ENDIF >= g(x) * h(B) x > 30 x <= 30 90*x >= g(x) * h(B) x*x >= g(x) * h(B) 90*x >= 2*x * h(B) x*x >= 0 * h(B) 90*x >= 2*x * 60 x*x >= 0 90*x >= 120*x

  11. Using a “Mechanical” Proof Checker (PVS) PVS SPECIFICATION OUTPUT FROM PVS THEOREM PROVER safe_prop : |------- {1} FORALL (B: bool, x: nat): f(x, B) >= g(x) * h(B) Rule? (grind) f rewrites f(x, B) to IF x > 30 AND B THEN x * x ELSE 90 * x ENDIF g rewrites g(x) to IF x <= 30 THEN 2 * x ELSE 0 ENDIF h rewrites h(B) to IF B THEN 60 ELSE 0 ENDIF Trying repeated skolemization, instantiation, and if-lifting, this simplifies to: safe_prop : {-1} x!1 >= 0 {-2} B!1 |------- {1} x!1 > 30 {2} 90 * x!1 >= 120 * x!1 Rule? demo_incorrect: THEORY BEGIN x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 60 ELSE 0 ENDIF safe_prop: THEOREM f(x,B) >= g(x) * h(B) END demo_incorrect

  12. Using a “Mechanical” Proof Checker (PVS) CORRECTED PVS SPECIFICATION OUTPUT FROM PVS THEOREM PROVER safe_prop : |------- {1} FORALL (B: bool, x: nat): f(x, B) >= g(x) * h(B) Rule? (grind) f rewrites f(x, B) to IF x > 30 AND B THEN x * x ELSE 90 * x ENDIF g rewrites g(x) to IF x <= 30 THEN 2 * x ELSE 0 ENDIF h rewrites h(B) to IF B THEN 40 ELSE 0 ENDIF Trying repeated skolemization, instantiation, and if-lifting, Q.E.D. Run time = 0.35 secs. Real time = 3.33 secs. demo_correct: THEORY BEGIN x: VAR nat B: VAR bool f(x,B): nat = IF x > 30 AND B THEN x*x ELSE 90*x ENDIF g(x): nat = IF x <= 30 THEN 2*x ELSE 0 ENDIF h(B): nat = IF B THEN 40 ELSE 0 ENDIF safe_prop: LEMMA f(x,B) >= g(x) * h(B) END demo_correct

  13. The Key Difference • Simulation/Testing only explores a small part of the state space • Formal Verification explores the entire state space The only way you can assure yourself that there are no safety-relevant bugs hidden in some dark corner of your software is to explore the entire state space.

  14. Can We Automate Verification? Verified! Program The Big V or Safety Props Buggy! Counter Example

  15. Kurt Goedel (1906-1978) Answer: NO!

  16. Why Can’t We Automate Verification? • Algebra (in general) is undecidable • (i.e. There is no terminating algorithm that can determine whether a formula is true) • But some sub-theories are decidable (e.g. Presburger arithmetic): develop automated decision procedures • While loops in a program lead to induction proofs • (Discovery of the induction invariant involves creativity: n: P(n) requires P(n)  Q1(n) …  Qj(n)) • Modeling the environment that the program interacts with often involves continuous mathematics (e.g. calculus, trig) hybrid models • Finite-state models can be automatically analyzed via model-checking (execution times can be exorbitant)

  17. Proof Checkers But we do use Proof Checkers! (aka Theorem Provers) -- because we make too many mistakes Why then do you call it a “theorem prover” if it can only check? Because it does help, it just can’t do the whole job.

  18. State Machine Analysis State = (x: nat, y: nat) Events = {ex,ey, em ,es}

  19. State Machine Analysis: (Proving Invariants) ex ex ex 0, 0 10,0 20,0 30,0 es ey ey ey ey ex ex ex 0,10 10,10 20,10 30,10 em em es 0,20 PROVE: For all reachable states (x,y) from (0,0): x  A and y  B

  20. State Machine Analysis: (Proving Invariants) • Theorem proving • deduction • human directed • Model checking • exhaustive search • automatic • state space must be finite

  21. State Machine Analysis: (Proving Invariants) ex ex ex 0, 0 10,0 20,0 30,0 es ey ey ey ey ex ex ex 0,10 10,10 20,10 30,10 em em es 0,20 PROVE: for all reachable states (x,y) from (0,0): x  A and y  B Counter-example: A = 30, B = 20 ex ex ex es (0,0) ----> (10,0) ----> (20,0) ----> (30,0) ----> (0,30)

  22. State Machine Analysis: (Proving Invariants) INVARIANT: For all reachable states (x,y) from (0,0) that x  max(A,B) and y  max(A,B) Proof Approach: Assume invariant holds in current state. Show each transition preserves the invariant. By induction true for all reachable states.

  23. SMV Version of State Machine MODULE main VAR x : 0..50; y : 0..50; event: {ex,ey,em}; ASSIGN init(x) := 0; init(y) := 0; next(x) := case event = ex & x <= AA - 10 : x + 10; event = ey & y <= BB - 10 : x; event = em & x > y & y > 0 : x - y; event = em & y > x & x > 0 : x; 1 : y; esac; next(y) := case event = ex & x <= AA - 10 : y; event = ey & y <= BB - 10 : y + 10; event = em & x > y & y > 0 : y; event = em & y > x & x > 0 : y - x; 1 : x; esac;

  24. SMV Execution Times Execution Times (secs) Range of State Space Variables

  25. Output From SMV Model Checker DEFINE AA := 40; BB := 50; invariant_1 := AG (x <= AA & y <= BB); invariant_2 := AG (x <= max_AB & y <= max_AB); -- specification invariant_1 is false -- as demonstrated by the following execution sequence state 1.1: x = 0 y = 0 event = em state 1.2: event = ey state 1.3: y = 10 state 1.4: y = 20 state 1.5: y = 30 state 1.6: y = 40 state 1.7: y = 50 event = em state 1.8: x = 50 y = 0 -- specification invariant_2 is true

  26. Different perspectives on the Joys of Theorem Proving Ms. Miller, tell the verification team I will be a little late with that proof. Just two more lemmas and the proof of the KB3D CD&R algorithm will be complete!

  27. What Does It Look Like In a Theorem Prover? x Y System n Z x: VAR posreal % positive real number n: VAR posnat % positive natural number Y = IF n > 15 THEN x ELSE n * x ENDIF Z = IF x  10 AND n  10 THEN 2 * x / n ELSE 10 / x ENDIF safe_prop: THEOREM Y * Z <= 200

  28. Traditional V&V Reviews Coverage Test cases FM in development process size proofs 106states deeper, conceptual Requirements 1010states High-level design shallow, thousands of cases 10100states Low-level design 10100100states(i.e. 10 million lines code) Code Executable Programming language such as C++ have complex, ill-defined semantics

  29. Formal Methods Does Not Bring Perfection! intended functionality • peer review • simulation • testing Requirements proof High Level Design proof Detailed Design proof Implementation Are the component models accurate? • peer review • simulation • testing physical devices

  30. Warning: some of our team members are a little sensitive You guys are both my witnesses…He insinuated that ZFC set theory is superior to type theory!

  31. The Research Challenge • Formal Methods are not a silver bullet: • Must be applied early in life cycle---not a band aid • Involves a significant learning curve • Each new application requires the development of new models and analysis techniques • Cost of using formal methods still high for some applications. • The software engineering community is dominated by the process management viewpoint (i.e., that careful management of software development process is the key to quality) • Historically, disasters must occur before major changes are made.

  32. EXTRA SLIDES

  33. What Does It Look Like In a Theorem Prover? sys_lem: {-1} Y = IF n > 15 THEN x ELSE n * x ENDIF {-2} Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF |------- [1] Y * Z <= 200 Rule? (case "n > 15") Case splitting on n > 15, this yields 2 subgoals

  34. What Does It Look Like In a Theorem Prover? sys_lem.1 {-1} n > 15 [-2] Y = IF n > 15 THEN x ELSE n * x ENDIF [-3] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF |------- [1] Y * Z <= 200 Rule? (assert) Simplifying, rewriting, and recording with decision procedures, This completes the proof of sys_lem.1 What Happened?Well, assuming n > 15 we have: Y = x Z = 10 / x

  35. What Does It Look Like In a Theorem Prover? sys_lem.2 [-1] Y = IF n > 15 THEN x ELSE n * x ENDIF [-2] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF {-3} n <= 15 |------- [1] Y * Z <= 200 Rule? (assert) Simplifying, rewriting, and recording with decision procedures, give us the other subgoal

  36. What Does It Look Like In a Theorem Prover? sys_lem.2 {-1} Y = n * x [-2] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF [-3] n <=15 |------- [1] Y * Z <= 200 Rule? (replace -1) (hide -1) sys_lem.2 [-1] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF [-2] n <=15 |------- [1]n*x* Z <= 200

  37. What Does It Look Like In a Theorem Prover? Rule? (case "x <= 10 AND n <= 10") Case splitting on x <= 10 AND n <= 10, this yields 2 subgoals sys_lem.2.1 {-1} x <= 10 AND n <= 10 [-2] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF [-3] n <=15 |------- [1] n * x * Z <= 200 Rule? (replace -1) Replacing using formula -1, this simplifies to

  38. What Does It Look Like In a Theorem Prover? sys_lem.2.1 {-1} Z = 2 * x / n [-2] n <= 10 [-3] x <= 10 |------- [1] n * x * Z <= 200 Rule? (replace -1) (hide -1) [-1] n <= 10 [-2] x <= 10 |------- {1} n * x * (2 * x / n) <= 200 Rule? (assert) hit it with the hammer

  39. What Does It Look Like In a Theorem Prover? sys_lem.2.1 [-1] n <= 15 [-2] x <= 10 |------- {1} n * x * (2 * x / n) <= 200 Rule? (mult-ineq -2 -2) {-1} x * x <= 10 * 10 [-2] n <= 15 [-3] x <= 10 |------- {1} n * x * (2 * x / n) <= 200 Rule? (assert) Simplifying, rewriting, and recording with decision procedures, This completes the proof of sys_lem.2.1. But nothing happens! hit it with the hammer again

  40. What Does It Look Like In a Theorem Prover? [-1] Z = IF x <= 10 AND n <= 10 THEN 2 * x / n ELSE 10 / x ENDIF {-2} NOT (x <= 10 AND n <= 10) {-3} n <= 15 |------- [1] n * x * Z <= 200 Rule? (replace -1) (hide -1) {-1} x >10 OR n > 10 {-2} n <= 15 |------- [1] n * x * 10 / x <= 200 Rule? (assert) This completes the proof of sys_lem2.2. This completes the proof of sys_lem.2. Q.E.D.

  41. Different Domains --> Different Verification Challenges x c(x,y) System y Need to formally reason about c(x,y) = y /[Sin2(x) + Cos2(x)] trigonometry c(x,y) = series calculus c(x,y) = d/dx (x3y + y5)/x

More Related