Bounded Probabilistic Model Checking with the Murphi Verifier

1. Bounded Probabilistic Model Checking with the Murphi Verifier Enrico Tronci Computer Science Department, University of Rome “La Sapienza”, Via Salaria 113, 00198 Roma, Italy,tronci@di.uniroma1.it, http://www.dsi.uniroma1.it/~tronci Joint work with: G. Della Penna, B. Intrigila, I. Melatti, M. Venturini Zilli FMCAD 2004: Formal Methods in Computer-Aided Design, Austin, Texas, USA, November 14-17, 2004

2. Goal • Given • a stochastic system (e.g. a randomized protocol) M and • a property  for M, • check automatically that M satisfies . For example, a Probabilistic Model Checker can be used to verify properties like: The probability that a message is not delivered after 0.1 seconds is less than 0.8

3. Low System (e.g. public info) Example High System (e.g. private info) NRL PUMP Statistically modulated ACK ACK Buffer LS HS Data Data The NRL Pump is a special purpose-device that forwards data from a low (security) level systemLS to a high (security) level system HS, but not conversely. Idea: LS ACK delay isprobabilistically based on a moving average of HS ACK delays. NRL Pump (Probabilistic) Properties: LS HS Ready to send Ready to receive Minimize information flow from HS to LS. Read Data Got ACK Send Data Send ACK Enforce reasonable performances, i.e.: <average ACK delay as seen from LS> = <average HS ACK delay> Waiting ACK Done

4. Probabilistic Model Checking • Probabilistic Model Checking (PMC) typically uses: • Markov Chains to represent the system M to be verified; • A probabilistic logic (e.g. PCTL) to represent the property  to be verified. 0.6 : the probability of reaching (from the initial state 1) state 3 is less than 0.2 (TRUE). PCTL: [tt U (s = 3)]  0.2 M 1 0.4 0.5 • : the probability of reaching (from the initial state 1) state 3 is less than 0.1 (FALSE). • PCTL: [tt U (s = 3)]  0.1 2 3 0.4 1.0 0.1 In fact, the probability of reaching state 3 is exactly 0.16.

5. Bounded Probabilistic Model Checking • Bounded Probabilistic Model Checking (BPMC) uses: • Markov Chains to represent the system M to be verified; • A probabilistic logic with bounded Untils (e.g. BPCTL) to represent the property  to be verified. As for BMC, BPMC is less expressive (and may be more efficient) than PMC. 0.6 M : the probability of reaching state 3 in at most 2 steps is less than 0.2 (TRUE). PCTL: [tt U2 (s = 3)]0.2 1 0.4 : the probability of reaching state 3 in at most 2 steps is at least than 0.1 (TRUE). PCTL: [tt U2 (s = 3)] 0.1 0.5 2 3 0.4 • : the probability of reaching state 3 in at most 1 step is at least 0.1 (FALSE). PCTL: [tt U1 (s = 3)] 0.1 1.0 0.1 In fact, the probability of reaching state 3 in at most 2 steps is exactly 0.16, whereas state 3 is not reachable in 1 step.

6. State of the Art Not surprising the main obstruction to probabilistic model checking is State Explosion Roughly speaking there are 4 approaches: • Linear Algebra and Sparse Matrices, e.g. as in ETMCC; • Symbolic, OBDD based, e.g. PRISM; • Symbolic, MTBDD(Multi Terminal Binary Decision Diagrams) based, e.g. ProbVerus; • Hybrid, using MTBDDs allowing reals in [0, 1] on terminal nodes to represent the transition matrix and linear algebra algorithms to carry out the computation (PRISM). Typically, PRISM hybrid approach is faster than a probabilistic model checker based only on MTBDDs and avoids the state explosion of a probabilistic model checker based only on sparse matrices. Since PRISM implements all of the above approaches we will compare our results with PRISMbest performances.

7. Hybrid Systems Analysis On (Discrete Time) Hybrid Systems we run quite soon into state explosion using ANY of the previously mentioned approaches (i.e. using PRISM). • Intuitively, here is why: • the number of states is too large to be handled using a linear algebra approach • the complex math computations entailed by the system dynamics makes life very hard for OBDDs or MTBDDs.

8. Finite Horizon Probabilistic Murphi (FHP-Murphi) • We have shown (CHARME’03) that an explicit approach can overcome such limitations. Namely, we have extended the Murphi verifier as follows (FHP-Murphi): • Added finite precision real numbers and probabilities; • Replaced the set of the initial states with a distribution of initial states; • Labeled Murphi rules with probabilities, so as to define Markov Chains; • Replaced Murphi deterministic invariant with a BPCTL formula of form [tt Uk p] stating that that the probability of reaching an error state in k steps is less than a given threshold . • As for FHP-Murphi performances: • when PRISM can handle the verification task PRISM is faster than FHP-Murphi, • however there are many verification tasks (typically those involving hybrid systems) feasible with FHP-Murphi and out of reach for PRISM because of state explosion.

9. Nested Untils Are Interesting FHP-Murphi can only handle finite horizon safety properties, that is BPCTL formuals of form [tt Uk p]. However there are many interesting properties that are not definable using formulas of form [tt Uk p], and thus cannot be handled by FHP-Murphi. For example, robustness and reliability properties are very useful when carrying out requirements analysis. Typically such properties have the following form: The probability of getting into an undesired system state and staying there for a long time, is low. More formally: There is a low probability of reaching (within k steps) an undesired state s, such that there is not an high probability of reaching (within h steps) a non-undesired state from s. Example: if a (probabilistic dining) philosopher hasnot eatensince a long time (undesired state), then he will soon eat, thus avoiding perishing for starvation. We need nested Until’s to define reliability properties: ( defines undesired states) [tt Uk ([tt Uh] 1)]0

10. Our Goal Using nested untils it is possible to express interesting properties (e.g. robustness, reliability). Moreover such properties are particularly important for hybrid systems. Thus it is worthwhile extending FHP-Murphi to full BPCTL. Remark 1. Of course PRISM can handle full BPCTL, however when handling hybrid systems it dies quite soon because of state explosion. Remark 2. Of course explicit PCTL (no bound on path lengths) would also be interesting, however it is computationally harder than BPCTL. Here we will show how FHP-Murphi can be extended so as to handle full BPCTL.

11. FHP-Murphi Verification Algorithm No-nested-untils FHP-Murphi uses a BF visit to carry out its computations. However, a BF visit is not effective when dealing with nested untils. Example. Suppose we want to check if s |= F holds, where: F = [tt Uk] ,  = [tt Uh] and  is an atomic proposition. . Using the no-nested-untils FHP-Murphi algorithm this requires to check that for all t reachable from s in at most k steps it holds t |= . Since  = [tt Uh] , this means that for allt reachable from s in at most k steps we need to start a (nested) BF visit to check if t |= [tt Uh] holds. We will show how, using a Depth First (DF) visit and a cache we can avoid most of these nested visits.

12. Easy Case: No Nested Untils 1 1 s8 s6 s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7  1 We want to verify if s0 |= [tt U2]0.5. Atomic formula  holds in states:s1 ,s4 ,s7. After visitig s0, the probability of reaching a state in which  holds is: 0

13. Easy Case: No Nested Untils 1 1 s8 s6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. After visitig s1 the probability of reaching a state in which  holds is: (1/3)*1 = (1/3).

14. Easy Case: No Nested Untils 1 1 s8 s6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. After visitig s2 the probability of reaching a state in which  holds is: (1/3) + (1/3)*[…]

15. Easy Case: No Nested Untils 1 1 s8 s6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. After visitig s4 the probability of reaching a state in which  holds is: (1/3) + (1/3)*[(1/2)*1 + …]

16. Easy Case: No Nested Untils 1 1 s8 S6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. After visitig s3 the probability of reaching a state in which  holds is: (1/3) + (1/3)*[(1/2)*1 + (1/2)*0]

17. Easy Case: No Nested Untils 1 1 s8 s6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. After visitig s5 the probability of reaching a state in which  holds is: (1/3) + (1/3)[(1/2)1 + (1/2)0] + (1/3)[…]

18. Easy Case: No Nested Untils 1 1 s8 s6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 s3 1/3 1 s1  1 s7 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. After visitig s5 the probability of reaching a state in which  holds is: (1/3) + (1/3)[(1/2)1 + (1/2)0] + (1/3)[1]

19. Easy Case: No Nested Untils 1 1 s8 s6  s5 1/3 1 1/2 s4  1 1/3 s2 s0 1/2 Never reached, since at distance 3 from s0. s3 1/3 1 s1  1 s7 Never reached, since  holds in s1. 1 We want to verify if s0 |= [tt U2]0.5 holds. Atomic formula  holds in states:s1 ,s4 ,s6. Finally, the probability of reaching a state in which  holds is: (1/3) + (1/3)[(1/2)1 + (1/2)0] + (1/3)[1] = (1/3) + (1/6) + (1/3) = (5/6)  0.5. Thus s0 |= [tt U2]0.5 holds.

20. Nested Untils (Use Cache) CACHE 1/2 s5 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 We want to verify if s0 |= F holds, where: F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Notation P(s, [Uk]) = the probability that (Uk) holds on a path starting from s.

21. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. DF visit of s3 completed. We first compute P(s3, [tt Uh]) and then P(s3, [Uk-3]). Both probabilities are stored in cache.

22. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. DF visit of s2 completed. P(s2, [Uk-2]) = 1*p3 computed and stored in cache. Remark No need to revisit s3, we can use p3 in cache.

23. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Visiting s1. P(s1, [Uk-1]) = (1/2) P(s2, [Uk-2]) + (1/2) P(s4, [Uk-2]) = (1/2)p2 + … Remark No need to revisit s2, we can use p2 in cache. We need P(s4, [Uk-2]).

24. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Completed DF visit of s4. P(s4, [Uk-2]) = 1* P(s3, [Uk-3]) = 1*p3. Remark No need to revisit s3, we can use p3 in cache.

25. Nested Untils (Use Cache) CACHE 1/2 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) b3 = P(s3, [tt Uh]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) p1 = P(s1, [Uk-1]) s5 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Completed DF visit of s1. P(s1, [Uk-1]) = (1/2)* P(s2, [Uk-2]) + (1/2) P(s4, [Uk-2]) = (1/2)p2 + (1/2)p4. Remark No need to revisit s2, s4, we can use p2, p4 in cache.

26. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) p1 = P(s1, [Uk-1]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Visiting s0. P(s0, [Uk]) = (1/2)* P(s1, [Uk-1]) + (1/2) P(s5, [Uk-1]) = (1/2)p1 + …. Remark No need to revisit s1, we can use p1 in cache. We need P(s5, [Uk-1]).

27. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) p1 = P(s1, [Uk-1]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Visiting s5. P(s5, [Uk-1]) = (1/2)* P(s4, [Uk-2]) + (1/2) P(s3, [Uk-2]) = (1/2)p4 + …. Remark No need to revisit s4, we can use p4 in cache. We need P(s3, [Uk-2]) (not in cache)

28. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) p1 = P(s1, [Uk-1]) q3 = P(s3, [Uk-2]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Visit s3 and compute P(s3, [Uk-2]). Remark This computation may use P(s3, [tt Uh]) (b3) in cache.

29. Nested Untils (Use Cache) CACHE 1/2 s5 b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) p1 = P(s1, [Uk-1]) q3 = P(s3, [Uk-2]) p5 = P(s5, [Uk-1]) 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Completed DF visit of s5. P(s5, [Uk-1]) = (1/2)* P(s4, [Uk-2]) + (1/2) P(s3, [Uk-2]) = (1/2)p4 + (1/2)q3. Remark No need to revisit s4, s3, we use p4, q3 in cache.

30. Nested Untils (Use Cache) CACHE b3 = P(s3, [tt Uh]) p3 = P(s3, [Uk-3]) p2 = P(s2, [Uk-2]) p4 = P(s4, [Uk-2]) p1 = P(s1, [Uk-1]) p6 = P(s6, [Uk-2]) q3 = P(s3, [Uk-2]) p5 = P(s5, [Uk-1]) p0 = P(s0, [Uk]) 1/2 s5 1/2 1/2 . . . s0 1 s4 s3 ….. 1/2 1/2 s1 1 1/2 s2 F  [Uk]0.4 ,  [tt Uh]0.9 ,  atomic formula. Completed DF visit of s0. P(s0, [Uk]) = (1/2)* P(s1, [Uk-1]) + (1/2) P(s5, [Uk-1]) = (1/2)p1 + (1/2)p5. Remark 1. No need to revisit s1, s5, we use p1, p5 in cache. Remark 2. The sketched procedure is carried out on all U-subformulas of the BPCTL formula to be verified.

31. Experimental Results • To assess effectiveness of our approach we run experiments with • Randomized protocols • Hybrid Systems

32. Randomized Protocols • For such experiments we used: • PZ: Pnueli-Zuck probabilistic dining philosophersprotocol, as found in PRISM distribution; • MPZ: PZ modified so as to meetquality-of-service requirements; • LR: Lehmann-Rabin probabilistic dining philosophersprotocol; • MLR: LR modified so as to meetquality-of-service requirements. Property: When an undesired state s is reached, then the system almost always reaches, from s and in a few steps, a non-error state. More formally: There is a low probability that we reach in d steps an undesired state s, and there is not a high probability of reaching in k steps from s, a non-error state. BPCTL form: [tt Uk (und[tt Uherr] 1)]0 (PZ, LR, MPZ, MLR). All experiments run on a Linux 500 MHz INTEL Pentium III with 2GB RAM. FHP-Murphi options: -500m (use 500 MB RAM) PRISM options: default

33. Randomized Protocols Results PZ LR MPZ MLR PRISM better on PZ, LR. FHP-Murphi better on MPZ, MLR

34. Hybrid Systems We experimented with verification of a turbogas control system (HSCC’03), given a probability distribution on the user demand. Because of state explosion, this verification is feasible onlywith FHP-Murphi (i.e. PRISM runs out of memory). Property: If the system reaches an undesired state, then with high probability it is able to return to a non-undesired state. More formally: There is a low probability that we reach an undesired state s, such that there is not a high probability of reaching a non-undesired state from s. BPCTL form:[tt Uk ([tt Uh] 1)]0 ( defines undesired states). All experiments run on a Linux 500 MHz INTEL Pentium III with 2GB RAM. FHP-Murphi options: -500m (use 500 MB RAM)

35. Gas Turbine System Disturbances: electric users, param. var, etc Settings Fuel Valve Opening FG102 Controller Gas Turbine (Turbogas) Vrot, Texh, Pel, Pmc Vrot: Turbine Rotation speed Texh: Exhaust smokes Temperature Pel: Generated Electric Power Pmc: Compressor Pressure

36. Hybrid System Results Probability: P(q, [tt Uk ([tt Uh] 1)]), q initial state, h = (k/100). MAX_D_U: Max speed of variation of user demand (KW/sec).

37. Conclusions Performances When PRISM terminates it is faster than FHP-Murphi. However, PRISM is not suitable for verification tasks involving (complex) math computations. Examples of such tasks are: analysis of hybrid systems, analysis of randomized protocols, performance analysis, etc. FHP-Murphi can handle such verification tasks when the horizon is not too large. Expressivness PRISM can handle BPCTL formulas and much more. Hard to model hybrid systems using PRISM. FHP-Murphi only handles BPCTL formulas. Hybrid systems are easily modeled using FHP-Murphi (since all C functions on real number available). Moral: If you have to handle (complex) math use FHP-Murphi, else use PRISM.

38. Future Work • Infinite Horizon • Continuous Markov Chains • …..