Metrics for real time probabilistic processes

1. Metrics for real time probabilistic processes Radha Jagadeesan, DePaul University Vineet Gupta, Google Inc Prakash Panangaden, McGill University Josee Desharnais, Univ Laval

2. Outline of talk • Models for real-time probabilistic processes • Approximate reasoning for real-time probabilistic processes

3. Discrete Time Probabilistic processes • Labelled Markov Processes For each state s For each label a K(s, a, U) Each state labelled with propositional information 0.3 0.5 0.2

4. Discrete Time Probabilistic processes • Markov Decision Processes For each state s For each label a K(s, a, U) Each state labelled with numerical rewards 0.3 0.5 0.2

5. Discrete time probabilistic proceses • + nondeterminism : label does not determine probability distribution uniquely.

6. Real-time probabilistic processes • Add clocks to Markov processes Each clock runs down at fixed rate r c(t) = c(0) – r t Different clocks can have different rates • Generalized SemiMarkov Processes Probabilistic multi-rate timed automata

7. Generalized semi-Markov processes. Each state labelled with propositional Information Each state has a set of clocks associated with it. {c,d} s {c} u t {d,e}

8. Generalized semi-Markov processes. Evolution determined by generalized states <state, clock-valuation> <s,c=2, d=1>Transition enabled when a clockbecomes zero {c,d} s {c} u t {d,e}

9. Generalized semi-Markov processes. <s,c=2, d=1> Transition enabled in 1 time unit <s,c=0.5,d=1> Transition enabled in 0.5 time unit {c,d} s {c} u t {d,e} Clock c Clock d

10. Generalized semi-Markov processes. Transition determines: a. Probability distribution on next states b. Probability distribution on clock values for new clocks {c,d} s 0.2 0.8 {c} u t {d,e} Clock c Clock d

11. Generalized semi Markov proceses • If distributions are continuous and states are finite: Zeno traces have measure 0 • Continuity results. If stochastic processes from <s, > converge to the stochastic process at <s, >

12. Equational reasoning • Establishing equality: Coinduction • Distinguishing states: Modal logics • Equational and logical views coincide • Compositional reasoning: ``bisimulation is a congruence’’

14. With continuous time

15. Alas!

16. Instability of exact equivalence Vs Vs

17. Problem! • Numbers viewed as coming with an error estimate. (eg) Stochastic noise as abstraction Statistical methods for estimating numbers

18. Problem! • Numbers viewed as coming with an error estimate. • Reasoning in continuous time and continuous space is often via discrete approximations. eg. Monte-Carlo methods to approximate probability distributions by a sample.

19. Idea: Equivalence metrics • Jou-Smolka, Lincoln-Scedrov-Mitchell-Mitchell Replace equality of processes by (pseudo)metric distances between processes • Quantitative measurement of the distinction between processes.

20. Criteria on approximate reasoning • Soundness • Usability • Robustness

21. Criteria on metrics for approximate reasoning • Soundness • Stability of distance under temporal evolution: ``Nearby states stay close '‘ through temporal evolution.

22. ``Usability’’ criteria on metrics • Establishing closeness of states: Coinduction. • Distinguishing states: Real-valued modal logics. • Equational and logical views coincide: Metrics yield same distances as real-valued modal logics.

23. ``Robustness’’ criterion on approximate reasoning • The actual numerical values of the metrics should not matter --- ``upto uniformities’’.

24. Uniformities (same) m(x,y) = |2x + sinx -2y – siny| m(x,y) = |x-y|

25. Uniformities (different) m(x,y) = |x-y|

26. Our results

27. Our results • For Discrete time models: Labelled Markov processes Labelled Concurrent Markov chains Markov decision processes • For continuous time: Generalized semi-Markov processes

28. Results for discrete time models

29. Results for continuous time models

30. Metrics for discrete time probablistic processes

31. Bisimulation • Fix a Markov chain. Define monotone F on equivalence relations:

32. Defining metric: An attempt Define functional F on metrics.

33. Metrics on probability measures • Wasserstein-Kantorovich • A way to lift distances from states to a distances on distributions of states.

34. Metrics on probability measures

35. Metrics on probability measures

36. Example 1: Metrics on probability measures Unit measure concentrated at x Unit measure concentrated at y m(x,y) x y

37. Example 1: Metrics on probability measures Unit measure concentrated at x Unit measure concentrated at y m(x,y) x y

38. Example 2: Metrics on probability measures

39. Example 2: Metrics on probability measures THEN:

40. Lattice of (pseudo)metrics

41. Defining metric coinductively Define functional F on metrics Desired metric is maximum fixed point of F

42. Real-valued modal logic

43. Real-valued modal logic Tests:

44. Real-valued modal logic (Boolean) q q

45. Real-valued modal logic

46. Results • Modal-logic yields the same distance as the coinductive definition • However, not upto uniformities since glbs in lattice of uniformities is not determined by glbs in lattice of pseudometrics.

47. Variant definition that works upto uniformities Fix c<1. Define functional F on metrics Desired metric is maximum fixed point of F

48. Reasoning upto uniformities • For all c<1, get same uniformity [see Breugel/Mislove/Ouaknine/Worrell] • Variant of earlier real-valued modal logic incorporating discount factor c characterizes the metrics

49. Metrics for real-time probabilistic processes

50. Generalized semi-Markov processes. Evolution determined by generalized states <state, clock-valuation> : Set of generalized states {c,d} s {c} u t {d,e} Clock c Clock d