1 / 37

On Statistical Model Checking of Stochastic Systems

On Statistical Model Checking of Stochastic Systems. Koushik Sen Mahesh Viswanathan Gul Agha University of Illinois at Urbana-Champaign. Problem. Given a probabilistic model M (e.g. Markov Chains) Given a CSL formula (with unbounded until)  = P <p [  1 U  2 ]

colleenz
Download Presentation

On Statistical Model Checking of Stochastic Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. On Statistical Model Checking of Stochastic Systems Koushik Sen Mahesh Viswanathan Gul Agha University of Illinois at Urbana-Champaign

  2. Problem • Given a probabilistic model M (e.g. Markov Chains) • Given a CSL formula (with unbounded until)  = P<p[1 U 2] Probability that a path satisfies 1 until 2 is less than p • Can we say M ² using statistical model-checking?

  3. Solution • Given a probabilistic model M (e.g. Markov Chains) • Given a CSL formula (with unbounded until)  = P<p[1 U 2] Probability that a path satisfies 1 until 2 is less than p • Can we say M ² using statistical model-checking? • Using Monte Carlo simulation of “finite paths” • Using a sequence of inter-related statistical hypothesis testing YES with some assumptions

  4. Model Assumption • Sample execution paths can be generated through discrete-event simulation • Execution paths are sequences of the form = s0! s1! s2! … where each si is a state of the model and ti2R>0 is the time spent in the state si before moving to the state si+1 • A probability space can be defined on the execution paths of the model in such a way that the paths satisfying any path formula in our concerned logic (CSL or PCTL), is measurable • The number of states of the system is finite t0 t1 t2

  5. Semi Markov Chains (Simple Model) • Semi Markov Chains (S,sI,P,Q,L) • S – finite number of states (let |S| = N) • sI – initial state • P : S £ S ! [0,1] – transition probability matrix • Q : S £ S ! (R¸ 0! [0,1]) – continuous cumulative probability distribution function • L : S ! 2AP – labeling function, where AP is the set of atomic propositions P(s,s’) gives the probability of transition from s to s’ Q(s,s’) gives the distribution over time for which a state remains in state s before moving to state s’ • Examples: network protocols with quantified non-determinism or randomized algorithms

  6. Continuous Stochastic Logic (CSL) •  ::= true | a | Æ | : | PQ p() •  ::=  U<t |  U  | X  where Q2 {<,>,¸,·} • P< 0.5(§full) • Probability that queue becomes full is less than 0.5 • P>0.98(: retransmit U receive) • Probability that a message is eventually received successfully without any need for retransmission is greater than 0.98

  7. Goal • Model check properties in CSL against SMC models • Main Contribution: • Statistically model-check formulas of the form P<p[1 U 2] against SMC • boils down to model-checking the formula against the underlying Markov Chain

  8. Relevant Part of Model and Logic • Markov Chain (S,sI,P,L) • S – finite number of states (let |S| = N) • sI – initial state • P : S £ S ! [0,1] – transition probability matrix • L : S ! 2AP – labeling function, where AP is the set of atomic propositions • Unbounded Until in CSL • P<p[1 U 2]

  9. r r q sI s1 s2 ERR OK 1-r 1-q 1-r Example 1 1 P<p[true U ERR] (i.e. P<p[§ERR])

  10. Bounded Until (Checking s ² P<p[§<t a]) • Given a simple Semi Markov Chain M • paths in this model are infinite • Want to check if s ² P<p[§<t a] • a being an atomic proposition • Given , , and 1(type I, type II error, and indifference region) • , is the probability that our statistical algorithm gives a wrong answer

  11. p Observation y ……. 1 0 Checking s ² P<p[§<t a] • Sample n paths from s • Each path is of the form • = s0! s1! s2! … ! sn • Sample a path until t0+t1+…+tn > t or a is satisfied • let f path satisfied §<ta • let y = f/n t0 t1 tn §<t a

  12. Bounded Until (Checking s ² P<p[§<t a]) • n is computed such that the following holds • Pr[Y/n < p | ¸ p+1] · • Pr[Y/n ¸ p | · p-1] · where Y ~ Binomial(n,)

  13. Unbounded Until • Given a simple Markov Chain M • assume paths in this model are infinite • Want to check if s ² P<p[§ a] • a being an atomic proposition • Sample n paths from s • what is the length of each path to be sampled?

  14. Unbounded Until • Given a simple Markov Chain M • assume paths in this model are infinite • Want to check if s ² P<p[§ a] • a being an atomic proposition • Sample n paths from s • what is the length of each path to be sampled? • Simple Strategy: Sample a path till we encounter a state satisfying “a” • what happens if there is a path whose any extension does not have a state satisfying “a”? non-termination

  15. a Simple Example of Non-termination q : a 1-q 1 1 : a A sample path takes me to this state: will never encounter a state satisfying “a”

  16. a Solution q : a 1-q 1 1 : a Use stopping probability of ps (user supplied) at every state: at any state stop sampling with probability ps

  17. a Modified Model ps : a ps q(1-ps) 1 : a ps (1-ps)(1-q) 1-ps : a 1-ps Theorem: If a path from any state s 2 S in the model M satisfies 1 U 2 with some probability, p, then a path sampled from the same state in the modified model M’ will satisfy the same formula with probability at least p(1−ps)N-1qN-1, where N = |S| and q is the smallest non-zero transition probability in the model M.

  18. a Observation 1: Introduce stopping probability ps to sample finite paths Modified Model ps : a ps q(1-ps) ps : a ps (1-ps)(1-q) 1-ps : a 1-ps Theorem: If a path from any state s 2 S in the model M satisfies 1 U 2 with some probability, p, then a path sampled from the same state in the modified model M’ will satisfy the same formula with probability at least p(1−ps)N-1qN-1, where N = |S| and q is the smallest non-zero transition probability in the model M.

  19. p Observation 0 1 y Not There Yet (in checking s ² P<p[§a] ) • Sample n paths from s • Each path is of the form • = s0! s1! s2! … ! sn • Sample a path until we stop • let f paths satisfy §a and y = f/n • Note that we can determine if a finite path satisfies § a • We cannot determine if a finite path satisfies : : (§ a) t0 t1 tn ……. ? ? ? § a

  20. Solution (for checking s ²P<p[§ a]) • Use ideas from numerical model checking technique Strue = {s 2 S | s ² a} Sfalse = {s 2 S | no path from s satisfies § a} S? = S - Strue – Sfalse • Theorem: Probability of reaching a state in Strue or Sfalse is 1

  21. p Observation y ……. 1 0 Solution (in checking s ² P<p[§a] ) • Sample n paths from s • Each path is of the form • = s0! s1! s2! … ! sn • Sample a path until we reach a state in Strue or Sfalse • let f paths satisfied §a • let y = f/n t0 t1 tn §a

  22. p Observation y ……. 1 0 Solution (in checking s ² P<p[§a] ) • Sample n paths from s • Each path is of the form • = s0! s1! s2! … ! sn • Sample a path until we reach a state in Strueor Sfalse • let f path satisfied §a • let y = f/n How to check if a state belongs to Sfalse or s ² P=0[§ a] ? t0 t1 tn §a

  23. Simple Situation (Coin Toss) • Given a biased coin • P[head] = p (unknown) • P[tail] = 1-p • Want to check if • P[head] = 0 (i.e. p =0)

  24. Simple Situation (Coin Toss) • Given a biased coin • P[head] = p (unknown) • P[tail] = 1-p • Want to check if • P[head] = 0 (i.e. p =0) • toss the coin n times • suppose all the outcomes are tail (i.e. y = x1 + … + xn / n = 0) • Can we say that P[head] = 0?

  25. Simple Situation (Coin Toss) • Given a biased coin • P[head] = p (unknown) • P[tail] = 1-p • Want to check if • P[head] = 0 (i.e. p =0) • toss the coin n times • suppose all the outcomes are tail (i.e. y = x1 + … + xn / n = 0) • Can we say that P[head] = 0? Yes • Provided the error in our decision is bounded by a respectable small number (say,  =  = 0.01) • Type I error = P[Y· y | p > 0] ·, • where Y ~ Binomial(n,p)

  26. Simple Situation (Coin Toss) • Given a biased coin • P[head] = p (unknown) • P[tail] = 1-p • Want to check if • P[head] = 0 (i.e. p =0) • toss the coin n times • suppose all the outcomes are tail (i.e. y = x1 + … + xn / n = 0) • Can we say that P[head] = 0? Yes • Provided the error in our decision is bounded by a respectable small number (say,  =  = 0.01) • Type I error = P[Y· y | p > 0] ·, • where Y ~ Binomial(n,p) • Problem: • cannot compute Type I error (cannot bound P[Y=0], where Y~Binomial(n,p) and p>0)

  27. Simple Situation (Coin Toss) • Given a biased coin • P[head] = p (unknown) • P[tail] = 1-p • Want to check if • P[head] = 0 (i.e. p =0) • toss the coin n times • suppose all the outcomes are tail (i.e. y = x1 + … + xn / n = 0) • Can we say that P[head] = 0? Yes • Provided the error in our decision is bounded by a respectable small number (say,  =  = 0.01) • Type I error = P[Y· y | p > 0] ·, • where Y ~ Binomial(n,p) • Problem: • cannot compute Type I error (cannot bound P[Y=0], where Y~Binomial(n,p) and p>0) • Solution: • can bound P[Y=0], if Y~Binomial(n,p) and p¸ • assume p does not lie in the range (0,), where 0 <  < 1 • type I error = P[Y· y | p ¸] · P[Y=0 | p = ]

  28. Simple Situation (Coin Toss) • Therefore, given  and , compute n such that • P[Y=0] ·, where Y~Binomial(n,). • Compute n samples x1, x2, … xn • Say, P[head] = 0 if x1+… + xn/n = 0 • Else, say P[head] > 0 • Note: type II error = P[Y>0 | p =0] = 0 <  • Nothing to worry

  29. Observation 2: Introduce  and assume that p does not lie in the range (0,) Simple Situation (Coin Toss) • Therefore, given  and , compute n such that • P[Y=0] ·, where Y~Binomial(n,). • Compute n samples x1, x2, … xn • Say, P[head] = 0 if x1+… + xn/n = 0 • Else, say P[head] > 0 • Note: type II error = P[Y>0 | p =0] = 0 <  • Nothing to worry

  30. Sub-task: check if s 2 Sfalse i.e. s ² P=0[§ a] • Use Observation 1 and Observation 2 • assume that Pr[§ a] in M’ does not lie in the range (0,2), where 2 is provided as input to the model-checker

  31. p=0 Observation 0 1 check if s 2 Sfalse i.e. s ² P=0 [§a] ) • Sample n paths from s • Each path is of the form • = s0! s1! s2! … ! sn • Sample a path until we stop • say s 2 P=0[§ a] if at least one path satisfies § a • if none of the paths satisfy § a, then say s ² P=0[§ a] t0 t1 tn ……. ? ? ? ? ? § a

  32. p p=0 Observation Observation 0 1 0 1 y Comparison between P<p[§ a] and P=0[§ a]

  33. Model-checking Other Operators • Essentially same as statistical model-checking techniques proposed in [Younes and Simmons CAV’02] and [Sen, Viswanathan, Agha CAV’04]

  34. Main Result Summarized • Our algorithm A takes as input • a stochastic model M, • a formula  in CSL, • error bounds  and , and • three other parameters 1, 2, and ps. • The result of model checking is denoted by A1,2,ps(M, ,,) • can be either true or false.

  35. Main Result Summarized Theorem: If the model M satisfies the following conditions • C1: For every subformula of the form P¸ p in the formula  and for every state s in M, the probability that a path from s satisfies  must not lie in the range [ (p-1-)/(1-),(p+1)/(1-)] • C2: For any subformula of the form 1 U 2 and for every state s in M, the probability that a path from s satisfies 1 U 2 must not lie in the range (0, 2/((1-ps)N-1qN-1)], where N is the number of states in the model M and q is the smallest non-zero transition probability in M • Then the algorithm provides the following guarantees • R1 : • Pr[A1,2,ps(M, ,,) = true | M 2] · • Pr[A1,2,ps(M, ,,) = false | M²] ·

  36. Optimizations • Caching of results • Discount Optimization • checking s 2 Sfalse is expensive • do not check s 2 Sfalse for every state in the path • check if a state s 2 Sfalse with probability pd

  37. Conclusion • Interesting idea showing that unbounded until can be model-checked statistically given certain assumptions about the model holds • Statistical model-checking has limitations in general • If we have to choose 1, 2, and ps small, then running time can be considerably high • However, if values of 1, 2, and ps are reasonable then running time is fast • Running time increases if we want to get better error bounds (,) • Running time increases if time bound in bounded until is large • There is always a model for which the approach does not work for both bounded and unbounded until • Advantages: • No need to store states: sample as required • Estimate probability (see FCS’05, QAPL’05, QEST’05) using Vesta tool.

More Related