+CH

1. motor ligand FAST +ATT -ATT flagellar motor R +CH 3 SLOW MCPs MCPs CW W W P P -CH 3 A A ~ ~ B Y ~ P Z ATP ADP ATP rpoH gene P P B Y i i Transcription Feedback s 32 mRNA - Feedforward Heat Translation - s s 32 32 Heat stabilizes hsp1 hsp2 Heat Feedback Transcription & Translation FtsH Proteases Lon DnaK GroL Chaperones GroS Robustness and Complexity John Doyle Control and Dynamical Systems BioEngineering Electrical Engineering Caltech

2. Collaborators and contributors(partial list) Theory:Parrilo, Carlson,Paganini, Papachristodoulo, Prajna, Goncalves, Fazel, Lall, D’Andrea, Jadbabaie,many current and former students, … Web/Internet: Low, Willinger,Vinnicombe,Kelly, Zhu,Yu, Wang, Chandy, Effros, … Biology: Csete,Yi, Arkin, Simon, AfCS, Borisuk, Bolouri, Kitano, Kurata, Khammash, El-Samad, Gross, Endelman, Sauro, Hucka, Finney, … Physics:Mabuchi, Doherty, Barahona, Reynolds, Asimakapoulos,… Turbulence: Bamieh, Dahleh, Bobba, Gharib, Marsden, … Engineering CAD:Ortiz,Murray, Schroder, Burdick, … Disturbance ecology: Moritz, Carlson, Robert, … Finance:Martinez, Primbs, Yamada, Giannelli,… Caltech faculty Other Caltech Other

3. For more details www.cds.caltech.edu/~doyle www.aut.ee.ethz.ch/~parrilo And thanks to Carla Gomes for helpful discussions.

4. Subthemes of this program • Scalability of algorithms and protocols • Large network and physical problems • Decentralized, asynchronous, multiscale • Computational complexity: P/NP/coNP • Approaches • Duality • Randomness • Workshop II part of this program • Workshop last week on “Phase Transitions of Algorithmic Complexity”

5. The Internet hourglass Applications Web FTP Mail News Video Audio ping napster Transport protocols TCP SCTP UDP ICMP Ethernet 802.11 Power lines ATM Optical Satellite Bluetooth Linktechnologies IP

6. The Internet hourglass Applications Web FTP Mail News Video Audio ping napster Transport protocols TCP SCTP UDP ICMP IP on everything Ethernet 802.11 Power lines ATM Optical Satellite Bluetooth Linktechnologies Everything on IP IP From Hari Balakrishnan

7. Towards a theory of the Internet • The well-known original design principles are a rudimentary “theory of the Internet.” • This is a nearly pure robustness theory (little else is being optimized). • Can we provide a “deep,” complete, and coherent theory of internetworking? (Like standard comms and controls.) • If we can’t say something systematic about the Internet protocols, we’re probably kidding ourselves about our ability to treat more complex problems. • Nevertheless this is just a “warm-up” for a theory of ubiquitous embedded software, protocols, and networks for real-time control of everything, everywhere.

8. Network protocols. Files HTTP TCP IP packets packets packets packets packets packets Routing Provisioning

9. Network protocols. HTTP TCP Vertical decomposition Protocol Stack IP Routing Provisioning

10. Network protocols. HTTP TCP IP Horizontal decomposition Each level is decentralized and asynchronous Routing Provisioning

11. “Breaks” standard communications and control theories. • Coherent, complete theory is missing but possible. First cut nearly done. • In what sense, if any, is this optimal? • What needs to be done to fix it? HTTP TCP Vertical decomposition IP Horizontal decomposition Routing Provisioning

12. Key elements of new theory • Primal/dual vertical and horizontal decomposition (Kelly et al, Low et al) • Source coding into mice and elephants. (Appears to be “universal” but needs more study.) • Congestion control for bandwidth utilization and minimal delay. Proofs use relaxations (but still handcrafted). • How bad is short path (low delay for mice) routing for elephants in a “well-provisioned” network? Conjecture: Not bad. • Vertical and horizontal integration can be made “nearly” optimal in an asymptotic sense. (In what sense?) • Lots of people here are working out details (the IPAM team!). Stay tuned.

13. Networking protocols • Multiscale physics • Biological networks • Business, finance, econ organization • Unifying theoretical framework? Vertical decomposition Horizontal decomposition

14. What’s next? • Scalable, integrated robustness analysis and software/protocol verification for hybrid control of nonlinear systems. • New extensions to robust control using sum-of-squares and semidefinite programming (SOS/SDP) offers extraordinary promise. • Already demonstrated on wide array of complex problems (controls, maxcut, quantum entanglement). • Potentially deep connections between verification and robustness. • Huge implications for biology and physics. • That’s the good news.

15. Communications and computing Store Communicate Compute Communicate Communicate

16. Sense Store Communicate Compute Communicate Communicate Act Environment

17. Control Computation Communication Communication Devices Devices Dynamical Systems

18. From Software to/from human Human in the loop To Software to Software Full automation Integrated control, comms, computing Closer to physical substrate Store Communicate Compute Communicate Communicate Computation • New capabilities & robustness • New fragilities & vulnerabilities Communication Communication Devices Devices Control Dynamical Systems

19. Good new, bad news, good news • Good: Powerful new capabilities enabled by “embedded, everywhere” • Bad: Frightening new potentials for massive cascading failure events • Good: Need for new math tools for verifying robustness of embedded networking. • Embedded: Ubiquitous, sensing, actuating • Networking: Connected, distributed, asynchronous DeborahWorld

20. This represents an enormous change, the impact of which is not fully appreciated Robustness and verifiability of highly autonomous control systems with embedded software is the central challenge Until recently, there were no promising methods for addressing this full problem Even very special cases have had limited theoretical support for systematic verification of robustness Everything has changed! Store Communicate Compute Communicate Communicate Computation • New capabilities & robustness • New fragilities & vulnerabilities Communication Communication Devices Devices Control Dynamical Systems

21. “Breaks” standard communications and control theories. • Duality as a method for decomposition • Distributed and asynchronous control • Other applications • Robustness analysis • A posteriori error bounds for PDEs HTTP TCP Vertical decomposition IP Horizontal decomposition Routing Provisioning

22. Robust hybrid/nonlinear systems theory of embedded networks? Linear theory plus bounds, with scalable algorithms. “Theory” without scalable algorithms. Hacking. (Scalable algorithms without theory.) Theory with scalable algorithms? Most research: Not scalable, no theory.

23. Provably robust, scalable protocols for control over embedded networks. Provably robust, scalable Internet protocols. Robustness verification of embedded control software/hardware. Hacking. Theory with scalable algorithms.

24. Key issues • Robustness/Fragility: Uncertainty in components, environment, and modeling, assumptions, and computational approximations • Verifiability: Short proofs of robustness • Complexity: Extreme, highly structured internal complexity is typically needed to produce verifiably robust behavior • Scarce resources: All tradeoffs are aggravated by efficiency and scarce resources

25. Robustness, evolvability/scalability, verifiability Typical design IP Evolvability Verifiability Robustness Ideal performance • Relative to“nominal” performance under ideal conditions, robust performance typically requires • greater internal complexity • some loss of nominal performance • Tradeoffs between robustness, evolvability, and verifiability seem less severe (e.g. IP)

26. Robustness, evolvability/scalability, verifiability Ideal performance Robustness Evolvability • That a system is not merely robust, but verifiably so, is an important engineering requirement and major research challenge • There is much anecdotal evidence and some new theoretical support as well for the compatibility of robustness, evolvability, and verifiability • Verifiability in forward engineering translates into comprehensibility in reverse engineering of biological systems • This research direction may be good news for understanding complex biological processes Verifiability

27. Computational complexity  • Assume you already know: • P/NP and NP complete • SAT and 3-SAT • …but not necessarily • NP vs coNP • Duality and relaxations

28. If true, there is always a short proof. • Which may be hard to find. Typically NP hard. 

29. Typically coNP hard. • Fundamental asymmetries* • Between P and NP • Between NP and coNP  • More important problem. • Short proofs may not exist. * Unless they’re the same…

30.  What makes a problem “harder”?

31. Easy to find solutions?  Satisfiable or feasible

32.  Easy to find proofs? Unsatisfiable or infeasible

33. 1 k k+1 0 Complexity?

34. Example: Satisfiability • SAT: Given a formula in propositional calculus, is there an assignment to its variables making it true? • We consider clausal form, e.g.: • (a OR (NOT b) OR c) AND(b OR d)AND(b OR (NOT d) OR a) • a, b, c, and d are Boolean (True/False) variables. • Problem is NP-Complete. (Cook 1971) • Shows surprising “power” of SAT for encoding computational problems.

35. Generating Hard Random Formulas • Key: Use fixed-clause-length model. • (Mitchell, Selman, and Levesque 1992) • Critical parameter: ratio of the number of clauses to the number of variables. • Hardest 3SAT problems at ratio = 4.3

36. Hardness of 3SAT 4000 50 var 40 var 20 var 3000 DP Calls Hard 2000 Easy 1000 Easy 0 2 3 4 5 6 7 8 Ratio of Clauses-to-Variables

37. 4000 50 var 40 var 20 var 3000 DP Calls 2000 1000 0 1.0 50% sat 0.8 0.6 • At low ratios: • few clauses (constraints) • many assignments • easily found • At high ratios: • many clauses • inconsistencies easily detected Probability 0.4 0.2 0.0 2 3 4 5 6 7 8 Ratio of Clauses-to-Variables Mitchell, Selman, and Levesque 1991 The 4.3 Point

38. 1.0 50% sat 0.8 4000 50 var 0.6 40 var 20 var 3000 Probability 0.4 DP Calls 2000 0.2 0.0 2 3 4 5 6 7 8 1000 Ratio of Clauses-to-Variables Mitchell, Selman, and Levesque 1991 0 • Refer to as a • SAT transition • Complexity transition • Is SAT transition either necessary or sufficient for complexity transition? • Connections with phase transitions in statistical physics? • Are transitions “sharp” in large size limit?

39. Theoretical Status Of Threshold • Very challenging problem ... • Current status: • 3SAT threshold lies between 3.45 and 4.6 (Motwani et al. 1994, Achlioptas et al. 2001, Kirousis 2002, Broder and Suen 1993, Dubois 2000; Achlioptas and Beame 2001, Friedgut 1997, etc.) • Other problems better characterized (NPP)

40. SAT Phase transitions ? ? Complexity

41. Quasigroups or Latin Squares A quasigroup is an n-by-n matrix such that each row and column is a permutation of the same n colors Quasigroup or Latin Square (Order 4) 32% preassignment Gomes and Selman 96

42. Quasigroup with Holes (QWH) 32% holes • Given a full quasigroup, “punch” holes into it • Always completable (satisfiable), so no SAT transition. • Appears to have a complexity transition (easy-hard-easy).

43. SAT Phase transitions ? ? Complexity

44. SAT Phase transitions ? ? Complexity Lots of problems with statistical physics story.

45. Why may it be reasonable that math, algorithms, and randomness are so effective? • Robust systems are verifiably so? • Do only robust systems persist as coherent, structured objects of study (universes, solar systems, planets, life forms, protocols, …)? • If so, then mostly robust (and verifiably so) systems are around for us to study.

46. Lattice models? What can we do with lattices that will be easy to understand, yet relevant to the “real” computational complexity problems that we most care about? • Key abstractions: • Robustness/Fragility • Verifiability • Complexity

47. Not connected Connected .2 .4 .6 .8 Density = fraction of occupied sites (black) Focus on “horizontal” paths.

48. “Vertical” paths in empty sites are allowed to connect through corners or edges. (8 neighbors) “Horizontal” paths connect only on edges. (4 neighbors.Ordinary square site percolation.) Focus on “horizontal” paths. Some (nonstandard) definitions

49. Critical phase transition at density = .59…

50. Not connected Connected .2 .4 .6 .8 Density = fraction of occupied sites (black) Focus on “horizontal” paths.