Stochastic Nonlinear Dynamics of Cellular Biochemical Systems:

1. Stochastic Nonlinear Dynamics of Cellular Biochemical Systems: Hong Qian Department of Applied Mathematics University of Washington

2. abstract I present the stochastic, chemical master equation as a unifying approach to the dynamics of biochemical reaction systems in a mesoscopic volume under a living environment. A living environment provides a continuous chemical energy input that sustains the reaction system in a nonequilibrium steady state with concentration fluctuations. We discuss nonlinear biochemical reaction systems such as phosphorylation-dephosphorylation cycle (PdPC) with bistability. Emphasis is paid to the comparison between the stochastic dynamics and the prediction based on the traditional approach based on the Law of Mass Action. We introduce the dirence between nonlinear bistability and stochastic bistability, the latter has no deterministic counterpart. For systems with nonlinear bistability, there are three dirent time scales: (a) individual biochemical reactions, (b) nonlinear network dynamics approaching to attractors, and (c) cellular evolution. For mesoscopic systems with size of a living cell, dynamics in (a) and (c) are stochastic while that with (b) is dominantly deterministic. Both (b) and (c) are emergent properties of a dynamic biochemical network; We suggest that the (c) is most relevant to major cellular biochemical processes such as epigenetic regulation, apoptosis, and cancer immunoediting. The cellular evolution proceeds with transitions among the attractors of (b) in a "punctuated equilibrium" manner.

3. An analytical theory for Darwin’s variations in mesoscopic scale? Intrinsic variations = Stochasticity Natural environmental selections = Bias

4. Here are some recent headlines:

5. Physics Chemistry Molecular Cellular Systems Evolutionary Biology

6. Stochastic Physical Chemistry (1940)

7. The Kramers’ theory and the CME clearly marked the domains of two areas of chemical research: (1) The computation of the rate constant of a chemical reaction based on the molecular structures, energy landscapes, and the solvent environment; and (2) the prediction of the dynamic behavior of a chemical reaction system, assuming that the rate constants are known for each and every reaction in the system.

8. Basic Facts on Single Molecule Stochastic Transition k1 A B k2 ‡ B A Time is in the waiting, the transition is instaneous!

9. The Biochemical System Inside Cells EGF Signal Transduction Pathway

10. The kinetic isomorphism between PdPC and GTPase

11. PdPC with a Positive Feedback

13. Gene regulatory and Biochemical signaling Networks

14. gene state 0 gene state 1 f g0 g1 TF synthesis k c ho[TF] degradation Another Kinetic Isophorphism k2 K + c E* k-2 † k1[K ] k0[K] E E* (A) k3[P] † K (B)

15. According to macroscopic chemical kinetics following the Law of Mass Action

16. NTP NDP E R R* P Pi Simple Kinetic Model based on the Law of Mass Action

17. hyperbolic delayed onset 1 activation level: f bistability 1 4 activating signal: q Bifurcations in PdPC with Linear and Nonlinear Feedback c = 0 c = 1 c = 2

18. According to mesoscopic chemical kinetics following the Chemical Master Equation

19. A Markovian Chemical Birth-Death Process k1(nx+1)(ny+1) k1nxny nZ k-1nZ k-1(nZ +1) k1 X+Y Z k-1

20. Chemical Master Equation Formalism for Chemical Reaction Systems • M. Delbrück (1940) J. Chem. Phys. 8, 120. • D.A. McQuarrie (1963) J. Chem. Phys. 38, 433. • D.A. McQuarrie, Jachimowski, C.J. & M.E. Russell (1964) Biochem. 3, 1732. • T.L. Hill & I.W. Plesner (1965) J. Chem. Phys. 43, 267; (1971) 54, 34. • I.G. Darvey & P.J. Staff (1966) J. Chem. Phys. 44, 990; 45, 2145; 46, 2209. • D.A. McQuarrie (1967) J. Appl. Prob. 4, 413. • G. Nicolis & A. Babloyantz (1969) J. Chem. Phys. 51, 2632. • R. Hawkins & S.A. Rice (1971) J. Theoret. Biol. 30, 579. • T.G. Kurtz (1971) J. App. Prob. 8, 344; (1972) J. Chem. Phys. 57, 2976. • Keizer (1972) J. Stat. Phys. 6, 67. • D. Gillespie (1976) J. Comp. Phys. 22, 403; (1977) J. Phys. Chem. 81, 2340.

22. and more recently …

23. 1-dimensional, 1-stable, 1-unstable fixed pts 1-dimensional, 2-stable, 1-unstable fixed pts 2-dimensional, 1-stable limit cycle via Hopf bifurcation

24. K R* R P NR* … (N-1)R* 0R* 1R* 2R* 3R* Markov Chain Representation v0 v1 v2 w0 w1 w2

25. Bistability and Emergent Sates defining cellular attractors Pk number of R* molecules: k

26. Landscape and Lyapunov Property

27. Extrema value q * q f(x,q)

28. the cusp the critical point q q2(e) q1(e) q *(e) (A) xss (B)

29. A fundamental difference of two types of landscapes • For a detailed balance system, such as protein folding dynamics, the energy landscape is given a priori. It directs the dynamics of the system. • For a system without detailed balance, f can be considered as an “landscape for dynamics”. However, it is a consequence of the dynamics. That is why we call it emergent. It dynamics is non-local. • Q: which one is the “fitness landscape”?

30. Biological Implications:for systems not too big, not too small, like a cell …

31. Emergent Mesoscopic Complexity • It is generally believed that when systems become large, stochasticity disappears and a deterministic dynamics rules. • However, this simple example clearly shows that beyond the “infinite-time” in the deterministic dynamics, there is another, emerging stochastic, multi-state dynamics! • This stochastic dynamics is completely non-obvious from the level of pair-wise, static, molecule interactions. It can only be understood from a mesoscopic, open driven chemical dynamic system perspective.

32. In a cartoon: Three time scales cy B A biochemical network t.s. molecular signaing t.s. fast nonlinear differential equations cx cellular evolution t.s. chemical master equation ny probability A discrete stochastic model among attractors emergent slow stochastic dynamics and landscape A B nx B appropriate reaction coordinate

33. Bistability in E. colilac operon switching Choi, P.J.; Cai, L.; Frieda, K. and Xie, X.S. Science, 322, 442- 446 (2008).

34. Bistability during the apoptosis of human brain tumor cell (medulloblatoma) induced by topoisomerase II inhibitor (etoposide) Buckmaster, R., Asphahani, F., Thein, M., Xu, J. and Zhang, M.-Q. Analyst, 134, 1440-1446 (2009)

35. Chemical basis of epi-genetics:Exactly same environment setting and gene, different internal biochemical states (i.e., concentrations and fluxes). Could this be a chemical definition for epi-genetics inheritance?

36. steady state chemical concentration distribution c2* c1* c2* concentration of regulatory molecules 2 c1* 2 The inheritability is straight forward: Note that f (x) is independent of volume of the cell, and x is the concentration!

37. (A) V=200 (C) (B) V=100 number of E * (D) concentration of E * time Ntot=100 Ntot=200

38. 1030 3x1022 1015 switching time in msec 3x107 1.0 0 500 1000 1500 2000 total number of molecule E 1011 yrs 10 hrs

39. Another insight

40. If one perturbs such a multi-attractor stochastic system: • Rapid relaxation back to local minimum following deterministic dynamics (level ii); • Stays at the “equilibrium” for a quite long tme; • With sufficiently long waiting, exit to a next cellular state.

41. The emergent cellular, stochastic “evolutionary” dynamics follows not gradual changes, but rather punctuated transitions between cellular attractors.

42. abrupt transition alternative attractor Relaxation process local attractor Relaxation, Wating, Barrier Crossing: R-W-BC of Stochastic Dynamics

43. Elimination • Equilibrium • Escape

44. An emerging “thermo”-dynamic structure in stochastic dynamics

45. The Thermodynamic Structure of Stochastic Systems

46. Two Origins of Irreversibility

47. Summary for Systems Biol. (1) As a physical chemistry approach to cellular biochemical dynamics, mesoscopic reaction systems can be modeled according to the CME: A new mathematical theory. (2) A possible chemical bases of epi-genetic inheritance is proposed; (3) Emerging landscape is introduced; (4) Beyond deterministic physics, there is stochastic diversity in evolutionary time!

48. Summary for Theoret. Physics (5) Nonlinear multi-attractors become stochastic attractors. Infinite large systems exhibit nonequilibrium phase transition with Maxwell construction and Lee-Yang theory; (6) A nonequilibrium statistical “thermo- dynamics” emerges from stochastic nonlinear dynamics; (7) Epigenetic switching is a form of nonequilibrium phase transition?

49. Thank You!