400 likes | 534 Views
Model Checking Durational Probabilistic Systems. Jeremy Sproston Università di Torino Joint work with François Laroussinie LSV, ENS de Cachan & CNRS UMR 8643 Institut d’Informatique, FUNDP, Namur 30/5/2005. Verifying probabilistic timed systems. Real-time systems :
E N D
Model Checking Durational Probabilistic Systems Jeremy Sproston Università di Torino Joint work with François Laroussinie LSV, ENS de Cachan & CNRS UMR 8643 Institut d’Informatique, FUNDP, Namur 30/5/2005
Verifying probabilistic timed systems • Real-time systems: • Quantitative information along paths • E.g. time-out occurs 30ms after a request • Probabilistic systems: • Quantitative information between paths • E.g. the packet is lost with probability 0.01 • In some systems, timing and probabilistic behaviour co-exist • Multimedia equipment, communication protocols, fault-tolerant systems
Verifying probabilistic timed systems • Aim: verify probabilistic timed systems • Properties: a request is followed by a response within 5ms with probability 0.99 or greater (soft deadlines) • System description formalism: • Durational probabilistic systems • Property description formalism: • PTCTL (Probabilistic Timed CTL) • E.g.: request P0.99(true U5 response) • Model checking • Focus on complexity issues
Context • Hansson and Jonsson [FACS94]: • Fully probabilistic systems • State-to-state transition corresponds to 1 time unit • Probabilistic timed temporal logic • De Alfaro [STACS97]: • Probabilistic-nondeterministic systems • State-to-state transition corresponds to 0 or 1 time units • Probabilistic timed temporal logic • Other work: • De Alfaro [CONCUR98, PhD thesis] • Andova et al. [FORMATS2003] • Kwiatkowska et al. [TCS2002] (PTCTL, probabilistic timed automata) • Baier et al. [IEEE TSE2003]
Fully probabilistic systems • Fully probabilistic system: FPS = (S,sinit,P,lab) • S is a set of states with initial state sinit • P: S x S [0,1] is a probabilistic transition matrix such that s’P(s,s’)=1 for all s • lab: S 2AP is a state labelling function fail 1 s3 State-to-state transition: probabilistic choice of target state according to the probability distribution over the state’s outgoing transitions. 0.01 try idle s0 s1 0.98 1 succ s2 1 0.01
Markov decision processes • Markov decision process: MDP = (S,sinit,,lab): • S is a set ofstates with the initial state sinit • S x Dist(S) is a probabilistic transition relation (elements are (s, ), for state s and distribution over states) • lab: S 2APis a state labelling function fail • State-to-state transition: • Nondeterministic choice • over the outgoing probability • distributions of the source state • Probabilistic choice of target • state according to the • distribution chosen in step 1. 1 s3 0.02 try idle s0 s1 0.98 1 succ s2 1 1
Markov decision processes • The coexistence of nondeterministic and probabilistic choice means that there may be no unique probability of certain behaviours • For example, we obtain the minimum and maximum probabilities of reaching a set of states fail • State-to-state transition: • Nondeterministic choice • over the outgoing probability • distributions of the source state • Probabilistic choice of target • state according to the • distribution chosen in step 1. 1 s3 0.02 try idle s0 s1 0.98 1 succ s2 1 1
Probabilistic CTL • Extend classical logics for model checking to reason about probabilistic behaviour • Probabilistic CTL (PCTL) • FPSs: Hansson and Jonsson [FAC94] • MDPs: Bianco and de Alfaro [FSTTCS95] • CTL with universal/existential path quantifiers replaced by a probabilistic path quantifier • This quantifier compares probability of exhibiting particular set of behaviours with a probability threshold
Probabilistic CTL • For example: • CTL: (safe U completed) • For all paths, the system will be safe until it has completed a task • PCTL on fully prob. systems: P0.9(safe U completed) • With probability 0.9 or greater, the system will be safe until it has completed a task • PCTL on Markov decision proc.s: P0.9(safe U completed) • For all ways of resolving nondeterministic choice, then with probability 0.9 or greater, the system will be safe until it has completed a task
PCTL model checking • For finite-state FPSs/MDPs, by induction on structure of formula, as for CTL • “Quantitative Until” (arbitrary probability), solve • Recursive linear equation(FPSs) [HJ94,CY95] • Linear optimisationproblem(MDPs)[BdA95] • Polynomial time algorithms • In practice: typically iterative solution methods • “Qualitative Until” (probability 1, 0): proceed by graph traversal[Vardi85]
Modelling (discrete) time • If state-to-state transitions take 0 or 1 time units, how to model a time-out of 100 time units? • Blow-up in the state space is proportional to the timing constraints of the system 100 transitions …
Modelling (discrete) time • If timing constraints are represented in binary, the size of the state space is exponential in the size of timing constraints • Can we avoid this? • Precedent for non-probabilistic timed systems: polynomial-time model checking for some timed temporal logic formulae (Laroussinie-Markey-Schnoebelen [FOSSACS2002]) • Aim: model checking algorithm for probabilistic timed systems which is polynomial in the size of the system
Durational probabilistic systems Revised draft Draft written Sub- mission Notif. reject New idea Draft written Sub- mission Notif. accept Publi- cation Final version
Durational probabilistic systems Revised draft [2,10] [0,30] [25,50] Draft written [0,60] Sub- mission Notif. reject [31,60] [25,50] New idea [25,50] Draft written [0,30] Sub- mission Notif. accept [7,30] [25,50] Publi- cation [50,110] Final version [10,28] [0,100]
Durational probabilistic systems Revised draft [2,10] [0,30] [25,50] Draft written [0,60] Sub- mission 0.3 Notif. reject [31,60] New idea 0.7 0.5 Draft written [0,30] Sub- mission Notif. accept [7,30] [25,50] 0.5 Publi- cation [50,110] Final version [10,28] [0,100]
Durational probabilistic systems (DPS) • Durational probabilistic system(Q,qinit,D,L) • Q - finite set of states • qinit – initial state • D Q x I x Dist(Q) – transition relation with duration and probabilistic branching • L: Q 2AP – labelling function • I is the set of finite intervals [n,m] (n,mnaturals) • Dist(Q) is the set of probability distributions over Q • AP is the set of atomic propositions
Semantics of DPSs • Transition of a DPS: (q,[n,m],) D • From state q, let d time units elapse (where d [n,m]), then select the next state probabilistically using the distribution • Semantics of DPSs: timed Markov decision processes (S,sinit,,lab) • S – finite state space, with initial state sinit • S x Naturals x Dist(S) (e.g. (s,d,)) • lab: S 2AP • Assume that in every loop in the graph of the DPS, at least 1 time unit must elapse
Semantics of DPSs • Jump semantics: a transition takes exactly d time units, and there are no intermediate states 0.5 (2) 0.5 Notif. reject Notif. reject q2 q2 0.5 0.5 Sub- mission Sub- mission (1) q1 q1 [1,3] Notif. accept q3 Notif. accept q3 0.5 0.5 (3) DPS Jump semantics 0.5 0.5
Semantics of DPSs • Continuous semantics: represent intermediate states explicitly; transitions have duration 1 (or 0) Continuous semantics (q1,0) 0.5 0.5 Notif. reject (1) (1) q2 0.5 Sub- mission q1 [1,3] (q1,1) 0.5 (q2,0) (1) q3 Notif. accept 0.5 0.5 (1) 0.5 (1) DPS (q3,0) (q1,2) 0.5
Semantics of DPSs • Consider the jump semantics of a DPS • The number of states of the jump semantics = number of states of the DPS • The number of transitions of the jump semantics is exponential in the (binary) encoding of the timing constraints • Consider the continuous semantics of a DPS • The number of states and transitions of the continuous semantics is exponential in the (binary) encoding of the timing constraints
PTCTL • Probabilistic Timed CTL: a probabilistic and timed extension of CTL • Based on PCTL and TCTL (Emerson et al. [Real-Time Systems92], Alur-Courcoubetis-Dill [I&C93]) • Includes P#(1 Uc 2) • Probabilistically quantified, timed until operator • Where #{<,,,>}, [0,1], {<,,=,,>}, c is a natural, and 1, and 2 are PTCTL formulae
PTCTL Example of the probabilistic timed until operator: • P(1 Uc 2) – the probability of reaching a 2-state via 1-states before c time units have elapsed is at least • Probability must be at least no matter how the nondeterministic choice of the DPS is resolved • A request is followed by a response within 5ms with probability 0.99 or greater request P0.99(true U5 response)
Model checking DPSs • First attempt: define the jump/continuous semantic TMDP of a DPS, and model check it • Require PTCTL model-checking algorithm for TMDPs • E.g. for P(1 Uc 2): for each state, want to compute the minimal probability of 1 Uc 2 • For each state, compute f(s,i), the minimal probability of 1 Ui 2 for all ic • Linear optimisation problem: f(s,i) := min(s,d,)s’ S (s’) . f(s’,i-d)
Model checking DPSs • Repeat similar idea for all PTCTL operators • Model checking the PTCTL formula for the TMDP M=(S,sinit, ,lab) runs in time O(|| . ((|S| . || . cmax) + poly(|M|))) where cmax is the largest timing constraint used in the formula
Model checking DPSs • Now model check the jump/continuous semantic TMDP of the DPS? • Jump semantics: || is exponential in the size of the timing bounds used in the DPS • Continuous semantics: |S| and || are exponential in the size of the timing bounds used in the DPS • But we can obtain solutions which are independent of the magnitude of the DPS’s timing constraints for the fragment of PTCTL in which time-subscripts using =c are disallowed (PTCTL[,])
Model checking DPSs • Solution for the jump semantics: • Represent the TMDP transitions for the left- and right-endpoints of timing intervals only • This is sufficient for verifying PTCTL (either go “as fast as possible” or “as slow as possible”) 0.5 Notif. reject (1) q2 q2 0.5 0.5 Sub- mission q1 q1 [1,300] 0.5 Notif. accept q3 q3 (300) 0.5 Jump semantics 0.5 DPS
Model checking DPSs • Solution for the continuous semantics: • Consider P(1 Uc 2) • Partition the state space according to the following general principles: • States are distinguished on whether they satisfy 1 or not; similarly with 2 • States are distinguished on whether a particular DPS transition can be taken or not • The number of classes in the resulting partition is polynomial in the size of the DPS and the size of P(1 Uc 2)
Model checking DPSs • Solution for the continuous semantics (continued): • Construct a TMDP from the partition • States = classes of the partition • Transitions: derived from the DPS • The TMDP suffices for model checking PTCTL against the continuous semantics
Model checking DPSs • Model checking the PTCTL formula for DPS = (Q,qinit,D,L) runs in time: • Jump semantics O(|| . ((|Q| . |D| . cmax) + poly(|DPS|))) • Continuous semantics: O((||3 . |D|3 . cmax) + poly(|| . |D| . |DPS|)))
Model checking DPSs • Considered model checking of DPSs against PTCTL[,] formulae • Polynomial time in the size of the DPS, but exponential in the size of the timing constraints of the formula • Can we do any better? • What are the lower bounds on the complexity of model checking PTCTL[,]?
NP-hardness of P(Fc p) • Consider P(Fc p) • p is an atomic proposition • (Fc p) is an abbreviation for (true Uc p) • K-th largest subset problem (NP-hard problem): • Finite set A = {a1, …, an} of natural numbers • Two integers K and B • Problem: do there exist at least K distinct subsets A’ A such that: aA’ a B
NP-hardness of P(Fc p) • Consider an instance of the K-th largest subset problem, with {a1, …, an}, K and B • Consider the following DPS • The only state labelled by p is qn • Let =K/2n • Then q0 satisfies P (FB p) if and only if the instance of K-th largest subset is positive [a1,a1] [a2,a2] [an,an] 0.5 0.5 0.5 … [0,0] [0,0] [0,0] q0 q1 q2 qn-1 qn 0.5 0.5 0.5
Complexity overview • PTCTL0/1: fragment of PTCTL with probability bounds 0 or 1 only • Fully probabilistic DPS: with no nondeterministic choice • Results in blue have a polynomial time algorithm in the size of the DPS
Conclusions • Obtained an algorithm running in polynomial time in the description of the DPS for PTCTL[,] • General picture of complexity results needs to be completed • Not considered in this talk: average-time-to-reach operator • Solution combines jump/continuous semantics constructions with the polynomial average-to-time-reach algorithm of de Alfaro [CONCUR98, PhD thesis]) • Applications to related models in continuous time: • Probabilistic timed automata • Continuous-time Markov decision processes
PCTL model checking of FPSs • Obtaining the characteristic set Sat(.): • Sat( P(1 U 2) ) ={s S | xs} where xs for all s S, are obtained from the recursive linear equation 0 if s Sno xs = 1 if s Syes s’ S P(s,s’) . xs’ if s S\(Sno Syes) and Syes – states that satisfy 1 U 2with probability exactly 1 Sno - states that satisfy 1 U 2with probability exactly 0
PCTL model checking of MDPs • The linear equation generalises tolinear optimisationproblems solvable iteratively, e.g. • Sat( P(1 U 2) ) ={s S | xs} 0 if s Sno xs = 1 if s Syes min Steps(s)s’ S(s’) . xs’if s S\(Sno Syes) and Syes – states that satisfy 1 U 2with min. prob. exactly 1 Sno - states that satisfy 1 U 2with min. prob. exactly 0
PCTL model checking of MDPs • The linear equation generalises tolinear optimisationproblems solvable iteratively, e.g. • Sat( P(1 U 2) ) ={s S | xs} 0 if s Sno xs = 1 if s Syes max Steps(s)s’ S(s’) . xs’if s S\(Sno Syes) and Syes – states that satisfy 1 U 2with max. prob. exactly 1 Sno - states that satisfy 1 U 2with max. prob. exactly 0
PCTL model checking of MDPs • The linear equation generalises tolinear optimisationproblems solvable iteratively, e.g. • Sat( P(1 U 2) ) ={s S | xs} 0 if s Sno xs = 1 if s Syes max Steps(s)s’ S(s’) . xs’if s S\(Sno Syes) • Cases for <, > follow similarly
PTCTL • Atomic propositions (New idea, Submission etc.) • Boolean connectives • P#(X ) – probabilistically quantified next operator • P#(1 Uc 2) – probabilistically quantified, timed until operator • D#() – average-time-to-reach operator where #{<,,,>}, [0,1], {<,,=,,>}, c is a natural, is a non-negative real, and , 1, and 2 are PTCTL formulae • Probabilistic quantifiers: Hansson and Jonsson 1994, Bianco and de Alfaro 1995 • Average-time-to-reach operator: de Alfaro 1997