610 likes | 821 Views
research supported by NSF. Stochastic Hybrid Systems: Modeling, analysis, and applications to networks and biology. João P. Hespanha Center for Control Engineering and Computation University of California at Santa Barbara. Talk outline. A model for stochastic hybrid systems (SHSs)
E N D
research supported by NSF Stochastic Hybrid Systems:Modeling, analysis, and applications to networks and biology João P. Hespanha Center for Control Engineeringand Computation University of Californiaat Santa Barbara
Talk outline • A model for stochastic hybrid systems (SHSs) • Examples: • network traffic under TCP • networked control systems • Analysis tools for SHSs • Lyapunov • moment dynamics • More examples … Collaborators: Stephan Bohacek (U Del), Katia Obraczka (UCSC), Junsoo Lee (Sookmyung Univ.),Mustafa Khammash (UCSB) Students: Abhyudai Singh (UCSB), Yonggang Xu (UCSB) Disclaimer: This is an overview, details in papers referenced…
Deterministic Hybrid Systems continuous dynamics guard conditions reset-maps q(t) 2 Q={1,2,…} ´ discrete state x(t) 2 Rn´ continuous state right-continuous by convention we assume here a deterministic system so the invariant sets would be the exact complements of the guards
Stochastic Hybrid Systems continuous dynamics transition intensities (probability of transition in interval (t, t+dt]) reset-maps q(t) 2 Q={1,2,…} ´ discrete state x(t) 2 Rn´ continuous state Continuous dynamics: Transition intensities: Reset-maps (one per transition intensity):
Stochastic Hybrid Systems continuous dynamics transition intensities (probability of transition in interval (t, t+dt]) reset-maps Special case: When all l are constant, transitions are controlled by a continuous-time Markov process q = 2 q = 1 specifiesq (independently of x) q = 3
Formal model—Summary State space: q(t) 2 Q={1,2,…} ´ discrete state x(t) 2 Rn´ continuous state Continuous dynamics: Transition intensities: # of transitions Reset-maps (one per transition intensity): • Results: • [existence] Under appropriate regularity (Lipschitz) assumptions, there exists a measure “consistent” with the desired SHS behavior • [simulation] The procedure used to construct the measure is constructive and allows for efficient generation of Monte Carlo sample paths • [Markov] The pair ( q(t), x(t) ) 2 Q£ Rn is a (Piecewise-deterministic) Markov Process (in the sense of M. Davis, 1993) [HSCC’04]
Stochastic Hybrid Systems with diffusion stochastic diff. equation transition intensities reset-maps Continuous dynamics: w´ Brownian motion process Transition intensities: Reset-maps (one per transition intensity):
Example I: Transmission Control Protocol network transmits data packets receives data packets server client r packets dropped with probability pdrop congestion control ´ selection of the rate r at which the server transmits packets feedback mechanism ´ packets are dropped by the network to indicate congestion
Example I: TCP congestion control network transmits data packets receives data packets server client r packets dropped with probability pdrop congestion control ´ selection of the rate r at which the server transmits packets feedback mechanism ´ packets are dropped by the network to indicate congestion TCP (Reno) congestion control: packet sending rate given by congestion window (internal state of controller) round-trip-time (from server to client and back) • initially w is set to 1 • until first packet is dropped, w increases exponentially fast (slow-start) • after first packet is dropped, w increases linearly (congestion-avoidance) • each time a drop occurs, w is divided by 2 (multiplicative decrease)
Example I: TCP congestion control # of packets already sent per-packet drop prob. pckts dropped per sec pckts sent per sec £ = TCP (Reno) congestion control: packet sending rate given by congestion window (internal state of controller) round-trip-time (from server to client and back) • initially w is set to 1 • until first packet is dropped, w increases exponentially fast (slow-start) • after first packet is dropped, w increases linearly (congestion-avoidance) • each time a drop occurs, w is divided by 2 (multiplicative decrease)
many SHS models for TCP… on-off TCP flows (no delays) long-lived TCP flows(no delays) on-off TCPflows with delay long-lived TCP flows with delay [SIGMETRICS’03]
Example II: Estimation through network process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • for simplicity: • full-state available • no measurement noise • no quantization • no transmission delays encoder logic ´ determines when to send measurements to the network decoder logic ´ determines how to incorporate received measurements
Stochastic communication logic process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • for simplicity: • full-state available • no measurement noise • no quantization • no transmission delays encoder logic ´ determines when to send measurements to the network decoder logic ´ determines how to incorporate received measurements
Stochastic communication logic process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • for simplicity: • full-state available • no measurement noise • no quantization • no transmission delays Error dynamics: prob. of sending data in[t,t+dt) depends on current error e reset error to zero
Stochastic communication logic process state-estimator white noise disturbance network x x(t1) decoder x(t2) encoder • with: • measurement noise • quantization • transmission delay error at encoder side based on data sent prob. of sending data in[t,t+dt) depends on current encoder error enec reset error to nonzero random variables error at decoder side based on data received [ACTRA’04]
Example III:And now for something completely different… Decaying-dimerizing chemical reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4 population of species S1 SHS model population of species S2 reaction rates Inspired by Gillespie’s Stochastic Simulation Algorithm for molecular reactions [Gillespie, 76]
Example III:And now for something completely different… Decaying-dimerizing chemical reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4 population of species S1 SHS model Disclaimer: Several other important applications missing. E.g., • air traffic control [Lygeros,Prandini,Tomlin] • queuing systems [Cassandras] • economics [Davis] population of species S2 reaction rates Inspired by Gillespie’s Stochastic Simulation Algorithm for molecular reactions [Gillespie, 76]
Generalizations of the SHS model 1. Stochastic resets can be obtained by considering multiple intensities/reset-maps One can further generalize this to resets governed by to a continuous distribution [see paper]
Generalizations of the SHS model -1 1 2. Deterministic guards can also be emulated by taking limits of SHSs barrier function e # 0+ g(x) The solution to the hybrid system with a deterministic guard is obtained as e # 0+ This provides a mechanism to regularize systems with chattering and/or Zeno phenomena…
Example: Bouncing-ball g g y c2 (0,1) ´ energy absorbed at impact y The solution of this deterministic hybrid system is only defined up to the Zeno-time t Zeno-time
Example: Bouncing-ball 1.2 1 0.8 0.6 0.4 0.2 1.2 0 1 -0.2 1.2 0 5 10 15 0.8 1 0.6 0.8 0.4 0.6 0.2 0.4 0 0.2 -0.2 0 5 10 15 0 -0.2 0 5 10 15 g g y c2 (0,1) ´ energy absorbed at impact e = 10–2 e = 10–3 e = 10–4 mean (blue) 95% confidence intervals (red and green)
Analysis—Lie Derivative Given scalar-valued function Ã: Rn£ [0,1) ! R derivative along solution to ODE Lfà Lie derivative of à One can view Lf as an operator space of scalar functions on Rn£ [0,1) space of scalar functions on Rn£ [0,1) à (x, t) LfÃ(x, t) Lf completely defines the system dynamics
Generator of a SHS continuous dynamics transition intensities reset-maps Given scalar-valued function Ã: Q £Rn£ [0,1) ! R generator for the SHS Dynkin’s formula (in differential form) where Lie derivative instantaneous variation reset term intensity diffusion term L completely defines the SHS dynamics Disclaimer: see following paper for technical assumptions [HSCC’04]
Generator of the SHS with diffusion Attention: These systems may have problems of existence of solution due to jumps! (stochastic Zeno) E.g. Given scalar-valued function Ã: Q £Rn£ [0,1) ! R Dynkin’s formula (in differential form) where no local solution for x(0) > 1 no global solution “jumping makes jumping more likely” “probability of multiple jumps in short interval not sufficiently small” L completely defines the SHS dynamics Disclaimer: see following paper for technical assumptions [HSCC’04]
Stochastic communication logics error dynamicsin remote estimation
Long-lived TCP flows long-lived TCP flows(with slow start) congestion avoidance slow-start
Lyapunov-based stability analysis error dynamics in NCS Dynkin’s formula Expected value of error: ) 2nd moment of the error: )
Lyapunov-based stability analysis error dynamics in NCS Dynkin’s formula Expected value of error: ) 2nd moment of the error: ) For constant rate: (e) = assuming (A–/2I) Hurwitz
Lyapunov-based stability analysis error dynamics in NCS Dynkin’s formula • One can show… • For constant rate: (e) = • E[e] ! 0 as long as > <[(A)] • E[||e||m] bounded as long as > 2 m<[(A)] • For polynomial rates: (e) = (e0Pe)kP > 0, k¸ 0 • E[e] ! 0 (always) • E[||e||m] bounded 8m both always true 8¸0 if A Hurwitz (no jumps needed for boundedness) getting more moments bounded requires higher jump intensities Moreover, one can achieve the same E[||e||2] with a smaller number of transmissions… [ACTRA’04, CDC’04]
Analysis—Moments for SHS state continuous dynamics transition intensities reset-maps z (scalar) random variable with mean and variance 2 Markov inequality (8e > 0) Tchebychev inequality Bienaymé inequality (8e > 0, a2R , n2N) often a few low-order moments suffice to study a SHS…
Polynomial SHSs continuous dynamics transition intensities reset-maps Given scalar-valued function Ã: Q £Rn£ [0,1) ! R generator for the SHS Dynkin’s formula (in differential form) where A SHS is called a polynomial SHS (pSHS) if its generator maps finite-order polynomial on x into finite-order polynomials on x Typically, when are all polynomials 8q, t
Moment dynamics for pSHS x(t) 2 Rn q(t) 2 Q={1,2,…} continuous state discrete state Monomial test function: Given for shortx(m) Uncentered moment: E.g,
Moment dynamics for pSHS monomial on x polynomial on x linear comb. of monomial test functions x(t) 2 Rn q(t) 2 Q={1,2,…} continuous state discrete state Monomial test function: Given for shortx(m) Uncentered moment: For polynomial SHS… ) ) linear moment dynamics
Moment dynamics for TCP long-lived TCP flows
Moment dynamics for TCP on-off TCP flows (exp. distr. files) long-lived TCP flows
Truncated moment dynamics 0.07 0.06 0.05 0.04 0.03 0.02 0.01 0 0 10 20 30 40 50 w Experimental evidence indicates that the (steady-state) sending rate is well approximated by a Log-Normal distribution PDF of the congestion window size w ) z Log-Normal r Log-Normal (on each mode) ) Data from: Bohacek, A stochastic model for TCP and fair video transmission, INFOCOM’03
Moment dynamics for TCP on-off TCP flows (exp. distr. files) long-lived TCP flows finite-dimensionalnonlinear ODEs
Long-lived TCP flows 0.1 0.02 0 1 2 3 4 5 6 7 8 9 10 200 sending rate mean 100 0 0 1 2 3 4 5 6 7 8 9 10 100 sending rate std. deviation 50 0 0 1 2 3 4 5 6 7 8 9 10 long-lived TCP flows with delay (one RTT average) moment dynamics in response to abrupt change in drop probability Monte Carlo (with 99% conf. int.) reduced model
Truncated moment dynamics (revisited) For polynomial SHS… linear moment dynamics Stacking all moments into an (infinite) vector 1 infinite-dimensional linear ODE In TCP analysis… lower ordermoments of interest approximated by nonlinear function of moments of interest that affect dynamics
Truncation by derivative matching infinite-dimensional linear ODE truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption: 1) and remain bounded along solutions to and 2) is (incrementally) asymptotically stable Theorem: 8> 0 9N s.t. if then class KL function Disclaimer: Just a lose statement. The “real” theorem is stated in [HSCC'05]
Truncation by derivative matching infinite-dimensional linear ODE truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption: 1) and remain bounded along solutions to and 2) is (incrementally) asymptotically stable Theorem: 8> 0 9N s.t. if then class KL function Proof idea: 1) N derivative matches ) & match on compact interval of length T 2) stability of A1) matching can be extended to [0,1)
Truncation by derivative matching infinite-dimensional linear ODE • Given , finding N is very difficult • In practice, small values of N (e.g., N=2) already yield good results • Can use to determine j(¢): k = 1 ! boundary condition on jk = 2 ! linear PDE on j truncated linear ODE (nonautonomous, not nec. stable) nonlinear approximate moment dynamics Assumption: 1) and remain bounded along solutions to and 2) is (incrementally) asymptotically stable Theorem: 8> 0 9N s.t. if then class KL function Proof idea: 1) N derivative matches ) & match on compact interval of length T 2) stability of A1) matching can be extended to [0,1)
Moment dynamics for DDR Decaying-dimerizing molecular reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4
Truncated DDR model Decaying-dimerizing molecular reactions (DDR): c3 c1 c2 S1 0 S2 0 2 S1 S2 c4 by matching for deterministic distributions [IJRC, 2005]
Monte Carlo vs. truncated model populations means 40 E[x1] 30 20 E[x2] 10 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Fast time-scale (transient) -3 x 10 populations standard deviations 4 Std[x1] 3 2 Std[x2] 1 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 -3 x 10 populations correlation coefficient -0.9 (lines essentiallyundistinguishable at this scale) [x2,x2] -0.95 -1 -1.05 -1.1 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 -3 x 10 Parameters from: Rathinam, Petzold, Cao, Gillespie, Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method. J. of Chemical Physics, 2003
Monte Carlo vs. truncated model populations means 40 30 E[x1] 20 E[x2] 10 Slow time-scale evolution 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 populations standard deviations 4 Std[x1] 3 2 Std[x2] 1 0 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 populations correlation coefficient error only noticeable when populations become very small(a couple of molecules, still adequate to study cellular reactions) 0.5 0 [x2,x2] -0.5 -1 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 Parameters from: Rathinam, Petzold, Cao, Gillespie, Stiffness in stochastic chemically reacting systems: The implicit tau-leaping method. J. of Chemical Physics, 2003
Conclusions • A simple SHS model (inspired by piecewise deterministic Markov Processes) can go a long way in modeling network traffic • The analysis of SHSs is generally difficult but there are tools available(generator, Lyapunov methods, moment dynamics, truncations) • This type of SHSs (and tools) finds use in several areas(traffic modeling, networked control systems, molecular biology, population dynamics in ecosystems)
Bibliography • M. Davis. Markov Models & Optimization. Chapman & Hall/CRC, 1993 • S. Bohacek, J. Hespanha, J. Lee, K. Obraczka. Analysis of a TCP hybrid model. In Proc. of the 39th Annual Allerton Conf. on Comm., Contr., and Computing, Oct. 2001. • S. Bohacek, J. Hespanha, J. Lee, K. Obraczka. A Hybrid Systems Modeling Framework for Fast and Accurate Simulation of Data Communication Networks. In Proc. of the ACM Int. Conf. on Measurements and Modeling of Computer Systems (SIGMETRICS), June 2003. • J. Hespanha. Stochastic Hybrid Systems: Applications to Communication Networks. In R. Alur, G. Pappas, Hybrid Systems: Computation and Control, LNCS 1993, pp. 387-401, 2004. • João Hespanha. Polynomial Stochastic Hybrid Systems. In Manfred Morari, Lothar Thiele, Hybrid Systems: Computation and Control, LNCS 3414, pp. 322-338, Mar. 2005. • J. Hespanha, A. Singh. Stochastic Models for Chemically Reacting Systems Using Polynomial Stochastic Hybrid Systems. To appear in the Int. J. on Robust Control Special Issue on Control at Small Scales, 2005. • A. Singh, J. Hespanha. Moment Closure Techniques for Stochastic Models in Population Biology. Oct. 2005. Submitted to publication. • Y. Xu, J. Hespanha. Communication Logic Design and Analysis for NCSs. ACTRA’04. • Y. Xu, J. Hespanha. Optimal Communication Logics for NCSs. In Proc. CDC, 2004. All papers (and some ppt presentations) available at http://www.ece.ucsb.edu/~hespanha
TCP window-based control Built-in negative feedback: r increases congestion r decreases RTT increases (queuing delays) w (window size) ´ number of packets that can remain unacknowledged for by the destination source f destination f e.g., w = 3 t0 1st packet sent t1 2nd packet sent t2 t0 3rd packet sent 1st packet received & ack. sent t1 2nd packet received & ack. sent t2 t3 1st ack. received )4th packet can be sent 3rd packet received & ack. sent t t w effectively determines the sending rate r: round-trip time