150 likes | 297 Views
Ymer: A Statistical Model Checker. H åkan L. S. Younes Carnegie Mellon University. Probabilistic Model Checking. Given a model M , a state s , and a property , does hold in s for M ? Model: stochastic discrete event system Property: probabilistic temporal logic formula
E N D
Ymer:A Statistical Model Checker Håkan L. S. Younes Carnegie Mellon University
Probabilistic Model Checking • Given a model M, a state s, and a property , does hold in s for M? • Model: stochastic discrete event system • Property: probabilistic temporal logic formula • Example: ≥0.1[≤5full] Ymer: A Statistical Model Checker
Statistical Solution Method • Use acceptance sampling to verify probabilistic properties • Hypothesis: P≥ [] • Observation: verify over a sample path • Bounds on probability of verification error • Probability of false negative: ≤ • Probability of false positive: ≤ Ymer: A Statistical Model Checker
Indifference region p1 p0 Error Bounds 2 Probability of errorwhen verifying P≥[] Actual probability of holding Ymer: A Statistical Model Checker
Ymer at a Glance • Supports time-homogeneous generalized semi-Markov processes • Limited to time-bounded properties • Distributed acceptance sampling (even with sequential acceptance sampling) • Purely statistical approach for verifying nested probabilistic statements Ymer: A Statistical Model Checker
DistributedAcceptance Sampling Slave Master Master Acceptance Sampling register model & property observation Slave simulation Slave simulation observation done Ymer: A Statistical Model Checker
Avoiding Sample Bias • Process observations as they come in? • No, bias against observations that take a long time to generate (long sample paths) • Process observations according to a predetermined schedule Schedule: 1 2 1 1 2 Received: 1 1 2 Ymer: A Statistical Model Checker
… Server Case Study:Symmetric Polling System • Single server, n polling stations • Stations are attended in cyclic order • Each station can hold one message • State space of size O(n·2n) Polling stations Ymer: A Statistical Model Checker
Results 100 Machine 1: 733 MHz Pentium III Machine 2: 500 MHz Pentium III 90 80 Percent of single machine 70 60 50 102 104 106 108 1010 1012 1014 Size of state space Ymer: A Statistical Model Checker
Nested Probabilistic Statements: Robot Grid World • Probability is at least 0.9 that goal is reached within 100 seconds while periodically communicating • ≥0.9[≥0.5[ ≤9 comm]≤100goal] Ymer: A Statistical Model Checker
Statistical Verification ofNested Probabilistic Statements • Cannot verify path formula without some probability of error • Probability of false negative: ≤ ′ • Probability of false positive: ≤ ′ Observation error Ymer: A Statistical Model Checker
Performance Considerations • Verification error is independent of observation error • Pick observation error to minimize effort • The same state may be visited along multiple sample paths • Memoize verification results to avoid repeated effort Ymer: A Statistical Model Checker
Robot Grid World (results) ≥0.9[≥0.5[ ≤9 comm]≤100goal] 104 103 = 0.025 = = 10−2 102 = 0.05 Verification time (seconds) 101 100 10−1 10−2 102 104 106 108 1010 1012 Size of state space Ymer: A Statistical Model Checker
Robot Grid World:Effect of Memoization 1.0 0.9 103 0.8 0.7 0.6 Unique/visited states Sample size 102 0.5 0.4 0.3 0.2 101 0.1 102 104 106 102 104 106 Size of state space Size of state space Ymer: A Statistical Model Checker
Availability • Source code is released under GPL • http://sweden.autonomy.ri.cmu.edu/ymer/ Ymer: A Statistical Model Checker