1 / 60

Stochastic Hybrid Systems: Modeling, analysis, and applications to networks and biology

research supported by NSF. Stochastic Hybrid Systems: Modeling, analysis, and applications to networks and biology. João P. Hespanha Center for Control Engineering and Computation University of California at Santa Barbara. Talk outline. A model for stochastic hybrid systems (SHSs)

waynes
Download Presentation

Stochastic Hybrid Systems: Modeling, analysis, and applications to networks and biology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. research supported by NSF Stochastic Hybrid Systems:Modeling, analysis, and applications to networks and biology João P. Hespanha Center for Control Engineeringand Computation University of Californiaat Santa Barbara

  2. Talk outline • A model for stochastic hybrid systems (SHSs) • Examples: • network traffic under TCP • networked control systems • Analysis tools for SHSs • Lyapunov • moment dynamics • More examples … Collaborators: Stephan Bohacek (U Del), Katia Obraczka (UCSC), Junsoo Lee (Sookmyung Univ.),Mustafa Khammash (UCSB) Students: Abhyudai Singh (UCSB), Yonggang Xu (UCSB) Disclaimer: This is an overview, details in papers referenced…

  3. Deterministic Hybrid Systems continuous dynamics guard conditions reset-maps q(t) 2 Q={1,2,…} ´ discrete state x(t) 2 Rn´ continuous state right-continuous by convention we assume here a deterministic system so the invariant sets would be the exact complements of the guards

  4. Stochastic Hybrid Systems continuous dynamics transition intensities (probability of transition in interval (t, t+dt]) reset-maps q(t) 2 Q={1,2,…} ´ discrete state x(t) 2 Rn´ continuous state Continuous dynamics: Transition intensities: Reset-maps (one per transition intensity):

  5. Stochastic Hybrid Systems continuous dynamics transition intensities (probability of transition in interval (t, t+dt]) reset-maps Special case: When all l are constant, transitions are controlled by a continuous-time Markov process q = 2 q = 1 specifiesq (independently of x) q = 3

  6. Formal model—Summary State space: q(t) 2 Q={1,2,…} ´ discrete state x(t) 2 Rn´ continuous state Continuous dynamics: Transition intensities: # of transitions Reset-maps (one per transition intensity): • Results: • [existence] Under appropriate regularity (Lipschitz) assumptions, there exists a measure “consistent” with the desired SHS behavior • [simulation] The procedure used to construct the measure is constructive and allows for efficient generation of Monte Carlo sample paths • [Markov] The pair ( q(t), x(t) ) 2 Q£ Rn is a (Piecewise-deterministic) Markov Process (in the sense of M. Davis, 1993) [HSCC’04]

  7. Stochastic Hybrid Systems with diffusion stochastic diff. equation transition intensities reset-maps Continuous dynamics: w´ Brownian motion process Transition intensities: Reset-maps (one per transition intensity):

  8. Example I: Transmission Control Protocol network transmits data packets receives data packets server client r packets dropped with probability pdrop congestion control ´ selection of the rate r at which the server transmits packets feedback mechanism ´ packets are dropped by the network to indicate congestion

  9. Example I: TCP congestion control network transmits data packets receives data packets server client r packets dropped with probability pdrop congestion control ´ selection of the rate r at which the server transmits packets feedback mechanism ´ packets are dropped by the network to indicate congestion TCP (Reno) congestion control: packet sending rate given by congestion window (internal state of controller) round-trip-time (from server to client and back) • initially w is set to 1 • until first packet is dropped, w increases exponentially fast (slow-start) • after first packet is dropped, w increases linearly (congestion-avoidance) • each time a drop occurs, w is divided by 2 (multiplicative decrease)

  10. Example I: TCP congestion control # of packets already sent per-packet drop prob. pckts dropped per sec pckts sent per sec £ = TCP (Reno) congestion control: packet sending rate given by congestion window (internal state of controller) round-trip-time (from server to client and back) • initially w is set to 1 • until first packet is dropped, w increases exponentially fast (slow-start) • after first packet is dropped, w increases linearly (congestion-avoidance) • each time a drop occurs, w is divided by 2 (multiplicative decrease)

  11. many SHS models for TCP… on-off TCP flows (no delays) long-lived TCP flows(no delays) on-off TCPflows with delay long-lived TCP flows with delay [SIGMETRICS’03]

  12. Example II: Estimation through network process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • for simplicity: • full-state available • no measurement noise • no quantization • no transmission delays encoder logic ´ determines when to send measurements to the network decoder logic ´ determines how to incorporate received measurements

  13. Stochastic communication logic process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • for simplicity: • full-state available • no measurement noise • no quantization • no transmission delays encoder logic ´ determines when to send measurements to the network decoder logic ´ determines how to incorporate received measurements

  14. Stochastic communication logic process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • for simplicity: • full-state available • no measurement noise • no quantization • no transmission delays Error dynamics: prob. of sending data in[t,t+dt) depends on current error e reset error to zero

  15. Stochastic communication logic process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • with: • measurement noise • quantization • transmission delay error at encoder side based on data sent prob. of sending data in[t,t+dt) depends on current encoder error enec reset error to nonzero random variables error at decoder side based on data received [ACTRA’04]

  16. Example III:And now for something completely different… Decaying-dimerizing chemical reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4 population of species S1 SHS model population of species S2 reaction rates Inspired by Gillespie’s Stochastic Simulation Algorithm for molecular reactions [Gillespie, 76]

  17. Example III:And now for something completely different… Decaying-dimerizing chemical reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4 population of species S1 SHS model Disclaimer: Several other important applications missing. E.g., • air traffic control [Lygeros,Prandini,Tomlin] • queuing systems [Cassandras] • economics [Davis] population of species S2 reaction rates Inspired by Gillespie’s Stochastic Simulation Algorithm for molecular reactions [Gillespie, 76]

  18. Generalizations of the SHS model 1. Stochastic resets can be obtained by considering multiple intensities/reset-maps One can further generalize this to resets governed by to a continuous distribution [see paper]

  19. Generalizations of the SHS model -1 1 2. Deterministic guards can also be emulated by taking limits of SHSs barrier function e # 0+ g(x) The solution to the hybrid system with a deterministic guard is obtained as e # 0+ This provides a mechanism to regularize systems with chattering and/or Zeno phenomena…

  20. Example: Bouncing-ball g g y c2 (0,1) ´ energy absorbed at impact y The solution of this deterministic hybrid system is only defined up to the Zeno-time t Zeno-time

  21. Example: Bouncing-ball 1.2 1 0.8 0.6 0.4 0.2 1.2 0 1 -0.2 1.2 0 5 10 15 0.8 1 0.6 0.8 0.4 0.6 0.2 0.4 0 0.2 -0.2 0 5 10 15 0 -0.2 0 5 10 15 g g y c2 (0,1) ´ energy absorbed at impact e = 10–2 e = 10–3 e = 10–4 mean (blue) 95% confidence intervals (red and green)

  22. Analysis—Lie Derivative Given scalar-valued function Ã: Rn£ [0,1) ! R derivative along solution to ODE Lfà Lie derivative of à One can view Lf as an operator space of scalar functions on Rn£ [0,1) space of scalar functions on Rn£ [0,1)   à (x, t) LfÃ(x, t) Lf completely defines the system dynamics

  23. Generator of a SHS continuous dynamics transition intensities reset-maps Given scalar-valued function Ã: Q £Rn£ [0,1) ! R generator for the SHS Dynkin’s formula (in differential form) where Lie derivative instantaneous variation reset term intensity diffusion term L completely defines the SHS dynamics Disclaimer: see following paper for technical assumptions [HSCC’04]

  24. Generator of the SHS with diffusion Attention: These systems may have problems of existence of solution due to jumps! (stochastic Zeno) E.g. Given scalar-valued function Ã: Q £Rn£ [0,1) ! R Dynkin’s formula (in differential form) where no local solution for x(0) > 1 no global solution “jumping makes jumping more likely” “probability of multiple jumps in short interval not sufficiently small” L completely defines the SHS dynamics Disclaimer: see following paper for technical assumptions [HSCC’04]

  25. Stochastic communication logics error dynamicsin remote estimation

  26. Long-lived TCP flows long-lived TCP flows(with slow start) congestion avoidance slow-start

  27. Lyapunov-based stability analysis error dynamics in NCS Dynkin’s formula Expected value of error: ) 2nd moment of the error: )

  28. Lyapunov-based stability analysis error dynamics in NCS Dynkin’s formula Expected value of error: ) 2nd moment of the error: ) For constant rate: (e) =  assuming (A–/2I) Hurwitz

  29. Lyapunov-based stability analysis error dynamics in NCS Dynkin’s formula • One can show… • For constant rate: (e) =  • E[e] ! 0 as long as  > <[(A)] • E[||e||m] bounded as long as  > 2 m<[(A)] • For polynomial rates: (e) = (e0Pe)kP > 0, k¸ 0 • E[e] ! 0 (always) • E[||e||m] bounded 8m both always true 8¸0 if A Hurwitz (no jumps needed for boundedness) getting more moments bounded requires higher jump intensities Moreover, one can achieve the same E[||e||2] with a smaller number of transmissions… [ACTRA’04, CDC’04]

  30. Analysis—Moments for SHS state continuous dynamics transition intensities reset-maps z (scalar) random variable with mean  and variance 2 Markov inequality (8e > 0) Tchebychev inequality Bienaymé inequality (8e > 0, a2R , n2N) often a few low-order moments suffice to study a SHS…

  31. Polynomial SHSs continuous dynamics transition intensities reset-maps Given scalar-valued function Ã: Q £Rn£ [0,1) ! R generator for the SHS Dynkin’s formula (in differential form) where A SHS is called a polynomial SHS (pSHS) if its generator maps finite-order polynomial on x into finite-order polynomials on x Typically, when are all polynomials 8q, t

  32. Moment dynamics for pSHS x(t) 2 Rn q(t) 2 Q={1,2,…} continuous state discrete state Monomial test function: Given for shortx(m) Uncentered moment: E.g,

  33. Moment dynamics for pSHS monomial on x polynomial on x linear comb. of monomial test functions x(t) 2 Rn q(t) 2 Q={1,2,…} continuous state discrete state Monomial test function: Given for shortx(m) Uncentered moment: For polynomial SHS… ) ) linear moment dynamics

  34. Moment dynamics for TCP long-lived TCP flows

  35. Moment dynamics for TCP on-off TCP flows (exp. distr. files) long-lived TCP flows

  36. Truncated moment dynamics 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 10 20 30 40 50 w Experimental evidence indicates that the (steady-state) sending rate is well approximated by a Log-Normal distribution PDF of the congestion window size w ) z Log-Normal r Log-Normal (on each mode) ) Data from: Bohacek, A stochastic model for TCP and fair video transmission, INFOCOM’03

  37. Moment dynamics for TCP on-off TCP flows (exp. distr. files) long-lived TCP flows finite-dimensionalnonlinear ODEs

  38. Long-lived TCP flows 0.1 0.02 0 1 2 3 4 5 6 7 8 9 10 200 sending rate mean 100 0 0 1 2 3 4 5 6 7 8 9 10 100 sending rate std. deviation 50 0 0 1 2 3 4 5 6 7 8 9 10 long-lived TCP flows with delay (one RTT average) moment dynamics in response to abrupt change in drop probability Monte Carlo (with 99% conf. int.) reduced model

  39. Truncated moment dynamics (revisited) For polynomial SHS… linear moment dynamics Stacking all moments into an (infinite) vector 1 infinite-dimensional linear ODE In TCP analysis… lower ordermoments of interest approximated by nonlinear function of  moments of interest that affect  dynamics

  40. Truncation by derivative matching infinite-dimensional linear ODE truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption: 1)  and  remain bounded along solutions to and 2) is (incrementally) asymptotically stable Theorem: 8> 0 9N s.t. if then class KL function Disclaimer: Just a lose statement. The “real” theorem is stated in [HSCC'05]

  41. Truncation by derivative matching infinite-dimensional linear ODE truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption: 1)  and  remain bounded along solutions to and 2) is (incrementally) asymptotically stable Theorem: 8> 0 9N s.t. if then class KL function Proof idea: 1) N derivative matches ) &  match on compact interval of length T 2) stability of A1) matching can be extended to [0,1)

  42. Truncation by derivative matching infinite-dimensional linear ODE • Given , finding N is very difficult • In practice, small values of N (e.g., N=2) already yield good results • Can use to determine j(¢): k = 1 ! boundary condition on jk = 2 ! linear PDE on j truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption: 1)  and  remain bounded along solutions to and 2) is (incrementally) asymptotically stable Theorem: 8> 0 9N s.t. if then class KL function Proof idea: 1) N derivative matches ) &  match on compact interval of length T 2) stability of A1) matching can be extended to [0,1)

  43. Moment dynamics for DDR Decaying-dimerizing molecular reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4

  44. Truncated DDR model Decaying-dimerizing molecular reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4 by matching for deterministic distributions [IJRC, 2005]

  45. Monte Carlo vs. truncated model populations means 40 E[x1] 30 20 E[x2] 10 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Fast time-scale (transient) -3 x 10 populations standard deviations 4 Std[x1] 3 2 Std[x2] 1 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 -3 x 10 populations correlation coefficient -0.9 (lines essentiallyundistinguishable at this scale) [x2,x2] -0.95 -1 -1.05 -1.1 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 -3 x 10 Parameters from: Rathinam, Petzold, Cao, Gillespie, Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method. J. of Chemical Physics, 2003

  46. Monte Carlo vs. truncated model populations means 40 30 E[x1] 20 E[x2] 10 Slow time-scale evolution 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 populations standard deviations 4 Std[x1] 3 2 Std[x2] 1 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 populations correlation coefficient error only noticeable when populations become very small(a couple of molecules, still adequate to study cellular reactions) 0.5 0 [x2,x2] -0.5 -1 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Parameters from: Rathinam, Petzold, Cao, Gillespie, Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method. J. of Chemical Physics, 2003

  47. Conclusions • A simple SHS model (inspired by piecewise deterministic Markov Processes) can go a long way in modeling network traffic • The analysis of SHSs is generally difficult but there are tools available(generator, Lyapunov methods, moment dynamics, truncations) • This type of SHSs (and tools) finds use in several areas(traffic modeling, networked control systems, molecular biology, population dynamics in ecosystems)

  48. Bibliography • M. Davis. Markov Models & Optimization. Chapman & Hall/CRC, 1993 • S. Bohacek, J. Hespanha, J. Lee, K. Obraczka. Analysis of a TCP hybrid model. In Proc. of the 39th Annual Allerton Conf. on Comm., Contr., and Computing, Oct. 2001. • S. Bohacek, J. Hespanha, J. Lee, K. Obraczka. A Hybrid Systems Modeling Framework for Fast and Accurate Simulation of Data Communication Networks. In Proc. of the ACM Int. Conf. on Measurements and Modeling of Computer Systems (SIGMETRICS), June 2003. • J. Hespanha. Stochastic Hybrid Systems: Applications to Communication Networks. In R. Alur, G. Pappas, Hybrid Systems: Computation and Control, LNCS 1993, pp. 387-401, 2004. • João Hespanha. Polynomial Stochastic Hybrid Systems. In Manfred Morari, Lothar Thiele, Hybrid Systems: Computation and Control, LNCS 3414, pp. 322-338, Mar. 2005. • J. Hespanha, A. Singh. Stochastic Models for Chemically Reacting Systems Using Polynomial Stochastic Hybrid Systems. To appear in the Int. J. on Robust Control Special Issue on Control at Small Scales, 2005. • A. Singh, J. Hespanha. Moment Closure Techniques for Stochastic Models in Population Biology. Oct. 2005. Submitted to publication. • Y. Xu, J. Hespanha. Communication Logic Design and Analysis for NCSs. ACTRA’04. • Y. Xu, J. Hespanha. Optimal Communication Logics for NCSs. In Proc. CDC, 2004. All papers (and some ppt presentations) available at http://www.ece.ucsb.edu/~hespanha

  49. END

  50. TCP window-based control Built-in negative feedback: r increases congestion r decreases RTT increases (queuing delays) w (window size) ´ number of packets that can remain unacknowledged for by the destination source f destination f e.g., w = 3 t0 1st packet sent t1 2nd packet sent t2 t0 3rd packet sent 1st packet received & ack. sent t1 2nd packet received & ack. sent t2 t3 1st ack. received )4th packet can be sent 3rd packet received & ack. sent t t w effectively determines the sending rate r: round-trip time

More Related