Introduction to Nonequilibrium Processes: some theoretical and computational issues

1. Introduction to Nonequilibrium Processes:some theoretical and computational issues For CSRC workshop on Advanced Monte Carlo Methods and Stochastic Dynamics June 21-25, 2011, Beijing, China Ping Ao Shanghai Center for Systems Biomedicine, Shanghai Jiao Tong University, China (上海交通大学系统生物医学研究院 敖平 ) aoping@sjtu.edu.cn Objectives: 1. Dealing with absence of detailed balance: new approach to stochastic processes. attempted by Boltzmann, Onsager, Prigogine, … 2. Unified framework for both near and far from equilibrium processes. deriving statistical mechanics and thermal dynamics (Ao, Comm. Theor. Phys., 2008) 3. Dynamical framework for complex systems: Darwinian dynamics, stressing uncertainty. Word equation of Darwin and Wallace (1858): Evolution by Variation and Selection third universal dynamics ? One of the principle objects of theoretical research in any department of knowledge is to find the point of view from which the subject appears in its greatest simplicity. Josiah Willard Gibbs (1839–1903)

2. Outstanding Questions on SM and TD Questions on foundation of statistical mechanics and thermodynamics (Uffink, 2005): 1) In what sense can thermodynamics said reduced to statistical mechanics? 2) How can one derive equations that are not time-reversal invariant from a time- reversal invariant dynamics? 3) How to provide a theoretical basis for the "approach to equilibrium" or irreversible processes? There are some brilliant successes in attempting those questions during past 150 years. A good recent example (2010) is the Fields Medal awarded to a set of beautiful results on Boltzmann equation. Success of statistical mechanics puzzles statisticians and mathematicians (Fine, 1973): 1) Irrelevant to inference and decision-making; 2) Assured by unstated methodological practices of censoring data and selective applying arguments; 3) A result of extraordinary good fortune. • Nonlinear, dissipative, no detailed balance, multiplicative noise • Construction with generalized Einstein relation

3. Development of Mathematical and Computational Methodologies on Stochastic Processes • New and powerful mathematical structure Arising from our study of stability of phage lambda genetic switch • Setting up the stage (only our work here) Potential in Stochastic Differential Equations: Novel Construction, P. Ao, J. Phys. A37 L25-L30 (2004). Structure of Stochastic Dynamics near Fixed Points, C. Kwon, P. Ao, and D.J. Thouless, Proc. Nat’l Acad. Sci. (USA) 102 (2005) 13029-13033. Existence and Construction of Dynamical Potential in Nonequilibrium Processes without Detailed Balance, L. Yin and P. Ao, J. Phys. A39 (2006) 8593-8601. On the Existence of Potential Landscape in the Evolution of Complex Systems, P. Ao, C. Kwon, and H. Qian, Complexity 12 (2007) 19-27. • In progress Global view of bionetwork dynamics: adaptive landscape. P. Ao. J. Genet. Genomics 36 (2009) 63-73 Stability analysis of a 4-d system, W. Xu, B. Yuan, P. Ao (submitted to CPL) Exact construction of potential function in a class of cyclic dynamics, R. Yuan, B. Yuan, P. Ao (in preparation)

4. Stochastic Differential EquationsBiological processes are fundamentally stochastic and non-conservative: Evolution by Variation and Selection, and physical processes are fundamentally conservative. Network dynamics: chemical reactions, for example dx/dt = f(x,t) + (x,t) (S) Gaussian and white noise: <  > = 0, < (X,t) (X,t’) > = 2 D (X,t) ε(t-t’) X = (X1, X2, … , Xn) Is there a quantity which specifies robustness and stability? A view not taken----Equation (S) can always be solved by a computer. Such view is in principle true, and can be useful. However, 1) Numerics can be time-consuming and uncertain (Ito vs Stratonovich); and 2) We, not computers, wish to understand Nature and to engineer.

5. Stochastic Dynamics Important but difficult subject in many fields • Biology: “ … the idea that there is such a quantity (adaptive landscape—P.A.) remains one of the most widely held popular misconceptions about evolution”. S.H. Rice, in Evolutionary Theory: mathematical and conceptual foundations (2004) • Chemistry: “The search for a generalized thermodynamic potential in the nonlinear range has attracted a great deal of attention, but these efforts finally failed.” G. Nicolis in New Physics, pp332 (1989) • Physics: “Statistical physicists have tried to find such a variational formulation for many years because, if it existed in a useful form, it might be a powerful tool for the solution of many kinds of problems. My guess … is that no such general principle exists.” J. Langer in Critical Problems in Physics, pp26 (1997) and, check recent issues of Physics Today, Physical Review Letters, … • Mathematics: gradient vs vector systems, unsolved (Holmes, 2005) dissipative, f  0 ; asymmetric, f  0 (absence of detailed balance): nonlinear ; stochastic with multiplicative noise • Economy (econophysics), finance, engineering, … . Potential function, Lyapunov function, cost function • Started a discussing group, with Prof. Bo Yuan of Computer Science Participating by faculty members in computer science, physics, mathematics, life science, … . Regularly about 40 students, under, master, PhD

6. Construction, I • Standard stochastic differential equation: dx/dt = f(x,t) + (x,t) (S) Gaussian and white, Wiener noise: < > = 0, <(x,t) (x,t’) > = 2 εD (x,t) (t-t’) • The desired equation (local and trajectory view): [S(x,t) + T(x,t)] dx/dt = - ψ(x,t) + (x,t) (N) Gaussian and white, Wiener noise: < > = 0, <(x,t)  (x,t’) > = 2 εS (x,t) (t-t’) which has4 dynamical elements: dissipative, transverse, driving, and stochastic forces. • The desired distribution (global and ensemble view): steady state (Boltzmann-Gibbs) distribution eq(x) ~ exp(- ψ(x) / ε )

7. Construction, II • Describingsame phenomenon [S(x,t) + T(x,t)] [f(x,t) + (x,t)] = - ψ(x,t) + (x,t) • Noise and deterministic “force” have independent origins [S(x,t) + T(x,t)] f(x,t) = - ψ(x,t) [S(x,t) + T(x,t)] (x,t) = (x,t) • Potential condition  [S(x,t) + T(x,t)] f(x,t) = 0, anti-symmetric, n(n-1)/2 equations • Generalized Einstein relation [S(x,t) + T(x,t)] D(x,t) [S(x,t) - T(x,t)] = S(x,t) symmetric, n(n+1)/2 equations • Total n2conditions for the matrix [S(x,t) + T(x,t)] ! Hence, S,T, ψ can be constructed from D and f (Inverse is easy).

8. Stochastic dynamical structure analysis (S(x.t) + T(x,t)) dx/dt = - grad ψ(x,t) + (x,t) (N) Four dynamical elements, the representation: S: Semi-positive definite symmetric matrix: dissipation; degradation T: Anti-symmetric matrix, transverse: oscillation, conservative ψ: Potential function: capacity, cost function, landscape : Stochastic force: noise Steady state distribution is determined by ψ(X), when independent of time, according to Boltzmann-Gibbs distribution, ρ ~ exp( - ψ / ε) ! Direct and quantitative measure for robustness and stability

9. Construction of Lyapunov Function • Standard stochastic differential equation: dx/dt = f(x) + (x,t) Gaussian and white, Wiener noise: < > = 0, <(x,t) (x,t’) > = 2 εD (x,t) (t-t’) (S) • The transformed equation: [S(x) + T(x)] dx/dt = - ψ(x) + (x,t) Gaussian and white, Wiener noise: < > = 0, <(x,t)  (x,t’) > = 2 εS (x,t) (t-t’) (N) • Potential condition:  X ( [S(x) + T(x)] f(x) ) = 0 ; Generalized Einstein relation: [S(x) + T(x)] D [S(x) - T(x)] = S(x); (Lyapunov equation) • ψ(x) is a Lyapunov function in whole state space. Proof: when ε = 0, dψ(x) /dt = d x/dt ·ψ(x) = - dx/dt [S(x) + T(x)] dx/dt = - dx/dt S(x) dx/dt ≤ 0 (S is non-negative by definition) ( < 0, not at an attractor; = 0, at an attractor, if S is positive definite ) Unsolved construction problem: “One could hope that a method for proving the existence of a Lyapunov function might carry with it a constructive method for obtaining this function. This hope has not been realized.” (Krasonvskii, 1959) quoted by P. Giesl, Construction of Global Lyapunov Functions Using Radial Basis Functions (Springer, 2007) More numerical construction methods motivated from engineering.

10. Ito, Stratonovich, and Present Methodon stochastic processes 1- d stochastic process (Ao, Kwon, Qian, Complexity, 2007) Stochastic differential equation (pre-equation according to van Kampen): • dt q = f (q) + ζ(q, t), Guassian-white noise, multiplicative Ito Process (I-type): • ∂tρ(q, t) = [ε∂q ∂q D(q) + ∂qD(q)φq(q) ] ρ(q, t) ρ(q, t = ∞) ~ exp{ − φ (q) /ε} / D(q) ; φ (q) = - ∫ dq f(q) / D(q) Stratanovich process (S-type): • ∂tρ(q, t) = [ε∂qD ½ (q)∂qD ½ (q) + ∂qD(q)φq(q) ] ρ(q, t) ρ(q, t = ∞) ~ exp{ − φ (q) /ε} / D ½ (q) Present process (A-type): • ∂tρ(q, t) = [ ε∂qD(q)∂q+ ∂qD(q)φq(q) ] ρ(q, t) ρ(q, t = ∞) ~ exp{ − φ (q) /ε}

11. Gradient Expansion Gradient expansion (Ao, 2004; 2005) Exact equations: ∂ × [G−1f(q)] = 0, G + Gτ= 2D. G ≡ (S + T ) -1 = D + Q Definition of F matrix from the “force” f in 2-d: • F11 = ∂1f1, F12 = ∂2f1, F21 = ∂1f2, F22 = ∂2f2 Lowest order gradient expansion—linear matrix equation: • GFτ− FGτ= 0 • G + Gτ= 2 D Q = (FD − DFτ ) / tr(F) • φ (q) = − ∫c dq · [G−1(q) f(q) ]

12. Where Are We? • A construction can be obtained generally. Outlined in Ao, physics/0302081; J. Phys. A (2004) • Linear case can be shown to be uniquely determined, with an explicit algorithm, irrespective of the stable or unstable nature of the dynamics. Kwon, Ao, and Thouless, PNAS (2005). • Equivalence to other methods Yin and Ao, J. Phys. A (2006). • Limit cycle Zhu, Yin, Ao, Intl J. Mod. Phys. B (2007) • Lambda switch: 2-dimensional (q.h.;w.l.;K-T; c.m.;…, physicists and mathematicians’ favorable dimension!) massless particle in magnetic field and in a potential • Gradient expansion: turned into a linear problem at each order

13. Towards New Theoretical Foundation of Evolutionary Biology (one of grand open problems in science, David Gross, Nobel laureate, 2010) • Solved two outstanding and fundamental problems in evolutional biology • Setting up the stage Laws in Darwinian evolutionary theory. P. Ao, Physics of Life Reviews 2 (2005) 116-156. Emerging of stochastic dynamical equalities and steady state thermodynamics from Darwinian dynamics. P. Ao. Commun. Theor. Phys. 49 (2008) 1073–1090. • In progress Global view of bionetwork dynamics: adaptive landscape. P. Ao. J. Genetics Genomics 36 (2009) 63-73. Static and dynamical characterisitics of 1-d Wright-Fisher process. SY Jiao, S. Xu, PY Jiang, FS Cui, B. Yuan, P. Ao, (in preparation)

14. Darwinian Dynamics and Adaptive Landscape • Word equation of Darwin and Wallace (1858): Evolution by Variation and Selection → dx/dt = f(x) + ζ(x, t), <ζ > = 0, <ζ(x,t) (x,t’) > = 2 D(x,t) (t-t’) • Two further fundamental and quantitative concepts: RA Fisher (1930): fundamental theorem of natural selection(fluctuation-dissipation theorem) S Wright (1932): adaptive landscape (the existence of potential function in stochastic processes) • The theoretical challenge: “In fact, there is no general potential function underlying evolution.All that we need to do in order to demonstrate this is find a case in which, under selectionalone, the allele frequencies in a population do not settle down to a stable point, but rathercontinue changing forever. We have already see an example of this in Figure 1.2. The factthat selection can result in limit cycles (see Figure 1.2B), in which the population repeatedlyrevisits the same states of some function that increases every generation. Note that this isnot a contradiction of the fundamental theorem, since the frequency-dependence that drivesthe fluctuations is part of the E(δw) term in Equation 1.52. Though evolutionary theory isnot built on the idea that any quantity is necessarily maximized, the idea that there is sucha quantity remains one of the most widely held popular misconceptions about evolution.” (Rice, S.H., Evolutionary Theory: mathematical and conceptual foundations.2004) • Meet the challenge: [S(x,t) + T(x,t)] dx/dt = grad (x,t) + (x,t) Wright evolutionary potential function (x,t) Gaussian and white: <  > = 0; < (x,t) (x,t’) > = 2 S(x,t) (t-t’) F-Theorem (Fisher’s FTNS, FDT in physical sciences)  (x) ~ exp{ (x)/ }, t   Ao, Physics of Life Reviews 2 (2005) 116-156. Limit cycle case: Zhu, Yin, Ao, 2006.

15. Global View of Bionetwork Dynamics: Adaptive Landscape. P. Ao, Journal of Genetics and Genomics (2009) Wright’s Adaptive LandscapeS. Wright, 1932 Waddington’s Developmental LandscapeC. Waddington, 1940 Genetic Switch as multiple equilibria, M. Delbruck,1949; Phage lambda genetic switch, Zhu, Yin, Hood, Ao, 2004 Neural computing landscape, J. Hopfield, 1982 Protein folding funnel landscape

16. How Would Nature Decide? “I assign more value to discovering a fact, even about the minute thing, than to lengthy disputations on the Grand Questions that fail to lead to true understanding whatever.” Galileo Galilei (1564-1642) First new experimental evidence: Influence of Noise on Force Measurements. Giovanni Volpe, Laurent Helden, Thomas Brettschneider, Jan Wehr, and Clemens Bechinger. PRL 104, 170602 (2010)

17. FIG. 1 (color online). (a) A Brownian particle (drawn not to scale) diffuses near a wall in the presence of gravitational and electrostatic forces. Its trajectory perpendicular to the wall is measured with TIRM. (b) Comparison of measured (bullets) and calculated (line) vertical diffusion coefficient as a function of the particle-wall distance. (c) Experimentally determined probability distribution of the local drift dz for dt = 5 ms at z = 380 nm (grey). The dashed line is a Gaussian in excellent agreement with the experimental data.

18. FIG. 4 (color online). Forces obtained from a drift-velocity experiment with added noise- induced drift [see Eq. (7) with alpha = 1 (open squares) (A-type), alpha = 0:5 (open triangles) (S-type), and alpha = 0 (open dots) (I-type)]. The solid squares represent the forces obtained from an equilibrium measurement (same as in Fig. 2).

19. Towards General Dynamical Framework:Darwinian dynamics as the candidate 1. Dealing with absence of detailed balance: new approach to stochastic processes. 2. Unified framework for both near and far from equilibrium processes. deriving statistical mechanics and thermal dynamics (Ao, Comm. Theor. Phys., 2008) 3. Dynamical framework for complex systems: Darwinian dynamics, stressing uncertainty. Word equation of Darwin and Wallace (1858): Evolution by Variation and Selection third universal dynamics One of the principle objects of theoretical research in any department of knowledge is to find the point of view from which the subject appears in its greatest simplicity. Josiah Willard Gibbs (1839–1903)

20. Acknowledgements Co-workers: • Lan Yin, Beijing Univ., Beijing, P.R. China • Xiaomei Zhu, GenMath, Seattle • David Thouless, Physics, Univ. Washington, Seattle, USA • Hong Qian, Applied Math, Univ. Washington, Seattle, USA • Chulan Kwon, Myongji Univ., S. Korea • Bo Yuan, Shanghai Jiao Tong University, China • Ruoshi Yuan, Shanghai Jiao Tong University, China • Yian Ma, Shanghai Jiao Tong University, China • Wei Xu, Shanghai Jiao Tong University, China Funding: • Institute for Systems Biology: Hood, Galas • USA NIH • China 985, 973 Thank you!