Stochastic Model Checking

1. Stochastic Model Checking 导师：李广元 报告人：喻灵婧

2. outline Introduction to stochastic model checking Discrete-time Markov chains (DTMC) reachability of DTMC Continuous-time Markov chains (CTMC) reachability of CTMC

3. outline Introduction to stochastic model checking Discrete-time Markov chains (DTMC) reachability of DTMC Continuous-time Markov chains (CTMC) reachability of CTMC

4. Why must we verify? • “Testing can only show the presence of errors, not their absence. ” • To rule out errors must consider all possible executions --often not feasible mechanically! • Learning from mistakes costly… – In May 2005, Toyota recalls about 75,000 cars. Some Prius drivers have reported sudden stalling or stopping at highway speeds. – According to reports “the stalling problem is due to a software glitch in its sophisticated computer system.” – Such problems are becoming more common: BMW 7 series, …Cost $?

5. Role of model checking • Automated techniques for the assurance of − safety − security, privacy & trust − performance − dependability • quantitative, as well as qualitative requirements: − how reliable is my car’s Bluetooth network? − how efficient is my phone’s power management policy? − is my bank’s web-service secure? • Focus on stochastic model checking − to capture probability and resource usage − range of quantitative analyses

6. Why probability? • When analysing system performance and dependability – to quantify arrivals, waiting times, time between failure, ... • When modelling uncertainty in the environment – to quantify imprecisions in system inputs – to quantify unpredictable delays, express soft deadlines, ... • When building protocols for networked embedded systems – randomized algorithms • When problems are undecidable deterministically – reachability of channel systems, ...

7. Illustrating examples • Security: Crowds protocol – analysis of probability of anonymity • IEEE 1394 Firewire protocol – proof that biased delay is optimal • Systems biology – probability that enzymes are absent within the deadline • Software in next generation of satellites – mission time probability (ESA project)

8. What is probabilistic model checking?

9. Probabilistic models • Other models: probabilistic variants of (priced) timed automata, or hybrid automata

10. outline Introduction to stochastic model checking Discrete-time Markov chains (DTMC) reachability of DTMC Continuous-time Markov chains (CTMC) reachability of CTMC

11. DTMC Formally, a DTMC is a tuple (S,sinit,P,L) with state space Sand state-labellingL and Pa stochastic matrix with P(s, s’) = one-step probability to jump from s to s Transitions between states occur in discrete time-steps

12. Probability measure on DTMCs • Events are infinite pathsin the DTMC D, i.e., Ω = Paths(D) – a path in a DTMC is just a sequence of states • A σ-algebra on D is generated by cylinder sets of finite paths πˆ: Cyl(πˆ) ={π ∈ Paths(D) | πˆis a prefix of π} – cylinder sets serve as basis events of the smallest σ-algebra on Paths(D) • Pr is the probability measure on the σ-algebra on Paths(D): Pr(Cyl(s0 . . . sn)) = ιinit(s0) ・ P(s0 . . . sn) – where P(s0 s1 . . . sn) =∏0i<nP(si, si+1) and P(s0) = 1, and – ιinit(s0) is the initial probability to start in state s0

13. outline Introduction to stochastic model checking Discrete-time Markov chains (DTMC) reachability of DTMC Continuous-time Markov chains (CTMC) reachability of CTMC

14. Reachability probabilities • What is the probability to reach a set of states B ⊆ S in DTMC D? • Which event does B mean formally? – the union of all cylinders Cyl(s0 . . . sn) where – s0 . . . snis an initial path fragment in D with s0, . . . , sn−1 /∈B and sn ∈ B

15. Reachability probabilities in finite DTMCs • Let Pr(s |= B) = Prs(B) = Prs{π ∈ Paths(s) | π |= B} – where Prsis the probability measure in D with single initial state s • Let variable xs= Pr(s |= B) for any state s – if B is not reachable from s then xs= 0 – if s ∈ B then xs= 1 • For any state s ∈ Pre∗(B) \ B:

16. Unique solution • Let D be a finite DTMC with state space S partitioned into: • S0= Sat(￢∃(C UB)) • S1 a subset of {s ∈ S | Pr(s |= C UB) = 1} that contains B • S? = S \ (S=0 ∪ S=1) • The vector Pr(s |= C UB)s∈S? is the uniquesolution of the linear equation system:

17. Reachability of DTMC example Model check: P>0.99 [ try U succ ] − Sat(try) = {s1}, Sat(succ) = {s3} − S0= {s0,s2} − S1= {s3} − S? = {s1} Linear equation system: A=(0.01) b=(0.98) − x0 = 0 − x1 = A · x1 + b − x2 = 0 − x3 = 1 Which yields: − Prob(try U succ) = x = [0, 98/99, 0, 1] − Sat(P>0.99 [ try U succ ]) = {s3}

18. Computing reachability probabilities(bounded) • The probabilities of the events C Un B can be obtained iteratively: x(0) = 0 and x(i+1) = Ax(i) + b for 0 i < n • where A =P(s, t) s,t∈C\B and b =P(s,B) s∈C\B • Then: x(n)(s) = Pr(s |= C Un B) for s ∈ C \ B

19. Reachability of DTMC example • Model check: − Sat(true) = {s0,s1,s2,s3}, Sat(succ) = {s3} − S0= − S1= {s3} − S? = {s0,s1,s2} • Linear equation system: • x(0) = 0 and x(i+1) = Ax(i) + b for 0 i < n.

20. Reachability of DTMC example • Which yields: − Prob(true U≤2succ) = x = [0.98, 0.9898, 0, 1] − Sat(P>0.98 [true U≤2succ ]) = {s1,s3}

21. outline Introduction to stochastic model checking Discrete-time Markov chains (DTMC) reachability of DTMC Continuous-time Markov chains (CTMC) reachability of CTMC

22. CTMC • Formally ，a CTMC is a tuple(S,sinit,R,L) with state space S and state-labellingL and R a transition rate matrix with 1-e-R(s, s’) *t= probability to jump from s to s’ triggered before t time units. 3/2 3/2 3/2 empty full s1 s2 s0 s3 3 3 3

23. outline Introduction to stochastic model checking Discrete-time Markov chains (DTMC) reachability of DTMC Continuous-time Markov chains (CTMC) reachability of CTMC

24. CTMC’s embeded DTMC • Embedded DTMC: emb(C)=(S,sinit,Pemb(C),L) − state space, initial state and labelling as the CTMC − for any s,s’S − E(s) is the exit rate of state s

25. Reachability of CTMC: time-bounded • Algorithm for checking Pr(s|=C UB)>J 1. If t=,then use approach for reachability(as in DTMC):solve a system of linear equations. 2. Make all states in S\C and B absorbing. 3. Uniformize the resulting CTMC with respect to its maximal rate. 4. Determine the transient probability at time t using s as initial distribution. 5. Return yes if transient probability of all-states lies in J, and no otherwise.

26. Reachability of CTMC example(time-bounded) P>0.65[ true U[0,7.5] full ] − “probability of the queue becoming full within 7.5 time units” State s3 satisfies full and no states satisfy ¬true − in C[full][¬true ∧¬ full] only state s3 made absorbing

27. Reachability of CTMC: time-unbounded until example Computing the summation of matrix-vector multiplications

28. Thank you!

29. CTMC:Transient and steady-state behaviour

30. CTMC:Computing transient probabilities

31. CTMC:Computing transient probabilities -uniformisation

32. CTMC:Computing transient probabilities -uniformisation • Intuition • Fix all exit rates to (at least) the maximal exit rate r occurring in CTMC C. • Thus, 1/r is the shortest mean residence time in the CTMC C. • Then normalize the residence time of all states with respect to r as follows: 1. replace an average residence time 1/r(s) by a shorter (or equal) one, 1/r 2. decrease the transition probabilities by a factor r(s)/r , 3. increase the self-loop probability by a factor (r−r(s))/r That is, slow down state s whenever r(s) < r.

33. CTMC:Computing transient probabilities -uniformisation

34. CTMC:Computing transient probabilities As P is a stochastic matrix(all entries in [0,1] & rows sum to 1), computing the matrix exponential Pi is numerically stable. Can truncate the summation using the techniques of Fox and Glynn [FG88], which allow efficient computation of the Poisson probabilities