1 / 43

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling. Introduction. Verify properties of discrete event systems Model independent approach Probabilistic real-time properties expressed using CSL Acceptance sampling Guaranteed error bounds. DES Example.

decker
Download Presentation

Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Probabilistic Verification of Discrete Event Systems using Acceptance Sampling

  2. Introduction • Verify properties of discrete event systems • Model independent approach • Probabilistic real-time properties expressed using CSL • Acceptance sampling • Guaranteed error bounds

  3. DES Example • Triple modular redundant system • Three processors • Single majority voter Voter CPU 1 CPU 2 CPU 3

  4. DES Example 3 CPUs upvoter up Voter CPU 1 CPU 2 CPU 3

  5. DES Example CPU fails 3 CPUs upvoter up 2 CPUs upvoter up 40 Voter CPU 1 CPU 2 CPU 3

  6. DES Example CPU fails repair CPU 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 40 19.2 Voter CPU 1 CPU 2 CPU 3

  7. DES Example CPU fails repair CPU CPU fails 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 1 CPU uprepairing CPU voter up 40 19.2 11 Voter CPU 1 CPU 2 CPU 3

  8. DES Example CPU fails repair CPU CPU fails CPU repaired 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 1 CPU uprepairing CPU voter up 2 CPUs upvoter up 40 19.2 11 4 … Voter CPU 1 CPU 2 CPU 3

  9. CPU fails repair CPU CPU fails CPU repaired 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 1 CPU uprepairing CPU voter up 2 CPUs upvoter up 40 19.2 11 4 … CPU fails repair CPU CPU repaired voter fails 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 3 CPUs up voter up 3 CPUs upvoter down 82 13 16 4 … Sample Execution Paths

  10. Properties of Interest • Probabilistic real-time properties • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up”

  11. Properties of Interest • Probabilisticreal-time properties • “The probability is at least 0.1 that the voter fails within 120 time units while at least 2 CPUs have continuously remained up” CSL formula: Pr≥0.1(“2 CPUs up”  “3 CPUs up” U≤120 “voter down”)

  12. Continuous Stochastic Logic (CSL) • State formulas • Truth value is determined in a single state • Path formulas • Truth value is determined over an execution path

  13. State Formulas • Standard logic operators: ¬, 1  2 … • Probabilistic operator: Pr≥() • True iff probability is at least  that  holds

  14. Path Formulas • Until: 1 U≤t2 • Holds iff 2 becomes true in some state along the execution path before time t, and 1 is true in all prior states

  15. Verifying Real-time Properties • “2 CPUs up”  “3 CPUs up” U≤120 “voter down” CPU fails repair CPU CPU fails CPU repaired 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 1 CPU uprepairing CPU voter up 2 CPUs upvoter up 40 19.2 11 4 … False!

  16. + + + ≤ 120 Verifying Real-time Properties • “2 CPUs up”  “3 CPUs up” U≤120 “voter down” CPU fails repair CPU CPU repaired voter fails 3 CPUs upvoter up 2 CPUs upvoter up 2 CPUs up repairing CPU voter up 3 CPUs up voter up 3 CPUs upvoter down 82 13 16 4 … True!

  17. Verifying Probabilistic Properties • “The probability is at least 0.1 that ” • Symbolic Methods • Pro: Exact solution • Con: Works only for restricted class of systems • Sampling • Pro: Works for any system that can be simulated • Con: Uncertainty in correctness of solution

  18. Our Approach • Use simulation to generate sample execution paths • Use sequential acceptance sampling to verify probabilistic properties

  19. Error Bounds • Probability of false negative: ≤ • We say that  is false when it is true • Probability of false positive: ≤ • We say that  is true when it is false

  20. Acceptance Sampling • Hypothesis: Pr≥()

  21. 1 –  Probability of acceptingPr≥() as true   Actual probability of  holding Performance of Test

  22. 1 –  False negatives Probability of acceptingPr≥() as true   False positives Actual probability of  holding Ideal Performance

  23. 1 –  False negatives Probability of acceptingPr≥() as true Indifference region   –    +  False positives Actual probability of  holding Actual Performance

  24. True, false,or anothersample? SequentialAcceptance Sampling • Hypothesis: Pr≥()

  25. Number ofpositive samples Number of samples Graphical Representation of Sequential Test

  26. Accept Number ofpositive samples Continue sampling Reject Number of samples Graphical Representation of Sequential Test • We can find an acceptance line and a rejection line given , , , and  A,,,(n) R,,,(n)

  27. Accept Number ofpositive samples Continue sampling Reject Number of samples Graphical Representation of Sequential Test • Reject hypothesis

  28. Accept Number ofpositive samples Continue sampling Reject Number of samples Graphical Representation of Sequential Test • Accept hypothesis

  29. Verifying Probabilistic Properties • Verify Pr≥() with error bounds  and  • Generate sample execution paths using simulation • Verify  over each sample execution path • If  is true, then we have a positive sample • If  is false, then we have a negative sample • Use sequential acceptance sampling to test the hypothesis Pr≥()

  30. Verification of Nested Probabilistic Statements • Suppose , in Pr≥(), contains probabilistic statements • Error bounds ’ and ’ when verifying 

  31. True, false,or anothersample? Verification of Nested Probabilistic Statements • Suppose , in Pr≥(), contains probabilistic statements

  32. Accept With ’ and ’ = 0 Number ofpositive samples Continue sampling Reject Number of samples Modified Test • Find an acceptance line and a rejection line given , , , , ’, and ’: A,,,(n) R,,,(n)

  33. With ’ and ’ > 0 Number ofpositive samples Number of samples Modified Test • Find an acceptance line and a rejection line given , , , , ’, and ’: Accept Continue sampling Reject

  34. Number ofpositive samples Number of samples Modified Test • Find an acceptance line and a rejection line given , , , , ’, and ’: A’,’,,(n) Accept ’ – ’= 1 – (1 – ( – ))(1 – ’) ’ + ’= ( + )(1 – ’) Continue sampling R’,’,,(n) Reject

  35. Verification of Negation • To verify ¬ with error bounds  and  • Verify  with error bounds  and 

  36. Verification of Conjunction • Verify 12  … n with error bounds  and  • Verify each i with error bounds  and /n

  37. Verification of Path Formulas • To verify 1 U≤t2 with error bounds  and  • Convert to disjunction • 1 U≤t2 holds if 2 holds in the first state, or if 2 holds in the second state and 1holds in all prior states, or …

  38. More on Verifying Until • Given 1 U≤t2, let n be the index of the first state more than t time units away from the current state • Disjunction of n conjunctions c1 through cn, each of size i • Simplifies if 1 or 2, or both, do not contain any probabilistic statements

  39. Performance  = 0.01  = 0.9 Average number of samples  =  = 10-10  =  = 10-1 Actual probability of  holding

  40. Performance  =  = 10-2  = 0.9 =0.001 =0.002 Average number of samples =0.005 =0.01 =0.02 =0.05 Actual probability of  holding

  41. Summary • Model independent probabilistic verification of discrete event systems • Sample execution paths generated using simulation • Probabilistic properties verified using sequential acceptance sampling • Easy to trade accuracy for efficiency

  42. Future Work • Develop heuristics for formula ordering and parameter selection • Use in combination with symbolic methods • Apply to hybrid dynamic systems • Use verification to aid controller synthesis for discrete event systems

  43. ProVer: Probabilistic Verifier http://www.cs.cmu.edu/~lorens/prover.html

More Related