From the Complex Systems Engineering Series:

1. From the Complex Systems Engineering Series: Emergent Behavior: What Is It, and How Should I Deal with It? INCOSE San Diego Presentation, 31 October 2009 Mark Halverson

2. Presentation Contents • Emergent Behavior: Background • Emergent Behavior: Definitions • It’s not that Traditional SE is Flawed, but………. • Emergent Behavior: Characteristics • Sources of Emergent Behavior • Management of Emergent Behavior • Examples • Advanced Emergent Behavior Management • Questions?

3. Emergent Behavior: Background • When my system did not behave completely as I had intended, I assumed that I had made a design or analysis error. • I was not aware of the difference between “complicated systems”, and “complex systems”. • Darn! I screwed up again. I’ll have to try harder next time. • After many years as a systems engineer, I became aware of “emergence”. • I was relieved to find out that with complex systems, it is (by definition) not possible to exactly model or predict emergent behavior. • It can not be mathematically modeled. • It will often dominate overall system performance. • No matter how smart you are, it is always good for a surprise.

4. Emergent Behavior: Background (cont.) • “There has been a lot of talk about emergence since it was ‘discovered’ as a subset of complexity theory in the 1980s, that discovery linking back to the beginning of systems theory in the 1920s.” [5] • “Emergent behavior forced its way into academic discussion with the advent of computer simulations. At first it was assumed that emergent behavior visible in the simulation results was due to imperfections in the simulation.” [6] • “Forensic engineers also had to deal with the concept of emergent behavior, which gave it a truly negative connotation.” [6]

5. Emergent Behavior: Definitions • “In philosophy, systems theory, as well as science, emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions.” [7] • “Emergent behavior is that which cannot be predicted through analysis at any level simpler than that of the system as a whole.” [8] George Dyson • “Emergent behavior, by definition, is what's left after everything else in a complex system has been explained.” [8] George Dyson

6. Emergent Behavior: Definitions (cont.) • “What distinguishes a complex system from a merely complicated one is that in a complex system, some additional behaviors and patterns emerge as a result of the patterns of relationship between the elements.” [9] • A “complicated system” may be very large, but it is always a linear sum of its parts. • A “complex system” is more than the sum of its parts, and shows behaviors that can not be traced directly to the behaviors of the individual parts. • Designing “complex systems” requires processes and techniques that go beyond traditional systems engineering.

7. It’s Not that TSE is Flawed……….. • Traditional systems engineering (TSE) is still the basis for our understanding of the art and science of systems engineering. • In many situations, traditional systems engineering will still be the best approach. • It is just that circumstances sometimes conspire against the use of TSE alone….………….

8. Emergent Behavior • Because emergent behavior plays a central role in the systems engineering of complex systems, this topic merits further examination………….

9. Emergent Behavior: Characteristics • Characteristics of Emergent Behavior: • Found in complex systems • Emergent behavior is usually sudden, unpredictable (small changes can cause very large differences in system behavior: chaos theory) • More difficult to predict and handle in closed-loop systems • Worsened when non-linearities are present • Often found where humans, or other adaptive (intelligent) system elements, are part of the higher-level system. This implies cognition and/or communication between the elements • Emergent behavior is especially tricky when the elements, or the structure of the elements, are time-varying • Emergent behavior is a surprise, often an unpleasant one • However, some emergence is beneficial, in fact, necessary

10. Emergent Behavior: Characteristics (cont.) • Traditional systems engineering is based upon linear systems theory • Superposition theory is valid for linear systems • Systems engineers like to divide and conquer in order to work on complexity at a more manageable level (decomposition) • Systems engineers like to study the behavior of the elements in order to understand the behavior of the system (reconstruction) • Beware! None of this is valid with non-linear (or complex) systems

11. Sources of Emergent Behavior • Sources of Emergent Behavior • Unwanted/unintended synchronization, or oscillation • Local or global networks allowing wanted/unwanted communication (often unintentional networks) • Non-linear interaction between simple elements • Thrashing: competition over a scarce resource • Chaotic (even if deterministic) behavior; Emergence is nature’s way of dealing with chaos • Intentional, or unintentional, feedback loops with poor gain margins • Intelligent, adaptive elements in the system (which means that the behavior of the elements as well as the system architecture varies with time, depending upon conditions)

12. Management of Emergent Behavior • Management strategies: • Prevent or mitigate the sources of emergence • Design limits into system to lessen the negative effects of emergence • Add extra stability and robustness to the system • Use simulations to detect and design for emergence (caution, very sophisticated simulations required; beware of chaos theory) • Reduce non-linearities • Increase scarce resources to minimize thrashing • Use the evolutionary (or incremental, or spiral) development life cycle to discover emergent behavior; promote its positive effects and suppress its negative effects

13. Emergent Behavior • Some Examples Will Make This Clear

14. Emergent Behavior: Examples • Automotive Welding Robots • The nature and quality of the weld depends upon the line voltage. No central control of individual robots. • Random irregularities/defects were observed. Traditional quality management techniques proved to be ineffective. • Found to be due to line voltage drops caused by simultaneous weldsfrom several robots. Design assumed random (non-synchronized) operation. • Networking: voltage line common to each robot. • Design change to each robot • Monitor line voltage, and wait until it is high enough. Increase the “intelligence” and adaptability of the robots. • Problem became worse, since this tended to increase the synchronization. Delay time non-linearity is the cause. • Further change was made to add random delays. This solved the problem. Note the importance of stochastic behavior.

15. Emergent Behavior: Examples • Freeway Traffic Jams • Most individual drivers want to avoid traffic jams, and maintain reasonable separation distance. No central control of each auto. • Observation: at high, but not saturated, traffic levels minor perturba-tions cause areas of light traffic and areas of traffic jams to appear with periodicity, for no apparent reason. Situation worst in the “fast lane”. Total effective flow rate of cars is significantly reduced. • Cause of emergent behavior • Individual drivers attempt to respond to minor perturbations, but with delay. Drivers in the fast lane have more “gain” in their behavior. • The delay time non-linearity coupled with the high gain led to oscillations; resulting in congestion & reduced flow rate. Similar to a mechanical servo with hysteresis in gears, and too high gain. • Solution: relax, go with the flow (i.e., reduce your driving “gain”). This will lead to variability in separation distances, so increase the average. This will only slightly decrease the net flow rate. Won’t work if the freeway is near saturation.

16. Emergent Behavior: Positive Example • Ant Colony • Individual ants have very simple behavior, and certainly lack the ability to plan and build a colony. There is no central control of each ant, but their individual behavior includes pre-programmed social behaviors. • Observation: large numbers of ants come together and build a useful colony. Size, shape, and architecture of the colony are not specifically predictable. But the process is robust: does not depend upon any one ant, and still succeeds if hundreds of ants are taken by predators. • Emergent behavior • Ant behavior responds to its immediate surroundings, often with stochastic components. This leads to adaptability and trial-and-error optimization; the colony adapts to a wide range of conditions. • Networking: colony pheromones communicate to all ants in the colony. This is communication, not control. • Robust: not dependent upon individual ants, or single communication links, or the total number of ants.

17. Emergent Behavior: Positive Example • Termite colonies are able to build large “cathedral” structures • Mere coexistence is not enough, local and global interaction between the termites is required to achieve emergence • No central plan, no intelligence required from the individual termites……..just simple individual behaviors • To ensure that the “cathedral” adapts to local conditions, a randomness to the individual’s behavior is necessary • Central, top-down control would actually suppress the positive aspects of emergence

18. Emergent Behavior: Positive Example • Optimized Sidewalk Structure • University of Michigan built a North Campus collection of buildings • Concrete sidewalks were to be placed in the Quad between buildings, but how to design? Once installed they are difficult to change. • Aesthetically designed sidewalks and landscaping were not useful in practice; students and faculty would often walk on the landscaping • Grass was planted, and students were allowed to walk as they wished. Later, concrete sidewalks were installed where the wear patterns were. • Emergent behavior • Students and faculty are cognitive and adaptive elements in a larger system. On the system level, their patterns of walking could not be accurately predicted. A later analysis showed that as a group, an optimum sidewalk structure was derived. • Individual behaviors, and a self-interest structure, produced emergence with positive results.

19. Emergent Behavior: Examples • London Millennium Foot Bridge • Analysis showed that the bridge was sufficiently strong and rigid. But it had to be shut down after opening due to strong lateral swinging when a sufficient number of pedestrians were walking on it. • Assessment: the natural lateral frequency of the bridge was close to the normal walking frequency of pedestrians. The pedestrians were becoming synchronized in phase and frequency to each other. The bridge designers did not anticipate this phenomenon. • Emergent behavior • It was discovered that humans on a swaying surface tend to subconsciously synchronize their footsteps to the sway. This takes place even with an imperceptibly small movement. • Networking: individual behavior responded to the common network (the swaying bridge), thereby resulting in an unexpected top-level system behavior. This is communication, not control.

20. Emergent Behavior: Examples • “Enterprise” Server Disk Drives • Disk drives are known to be sensitive to vibration, especially when it is synchronized to its seek activity. When several disk drives were mounted together, data faults were experienced. • Assessment: the computer system database searches caused several disk drives to seek simultaneously, thereby building up synchronized vibration that disturbed each other’s operation. • Emergent behavior • Simultaneous disk drive seek operations were inherent to the large enterprise servers, which induced emergent behavior among the multiple disk drives. The disk drives had to be especially designed for such applications. That is why they are much more expensive than “ordinary” disk drives. • Networking: The computer chassis communicated the seek motion from one disk drive to the others.

21. Emergent Behavior: Ultimate Example • The Human Brain • Made up of billions of neurons, each of which exhibits very simple behavior, and single-bit memory. Some stochastic characteristics. • Assessment: the top-level system (the brain, three pounds) displays an infinitely sophisticated and complicated behavior: language, visual, aural, and tactile I/O, the arts, culture, emotions, as well as logical thought and processing. Robust, adaptive, innovative. • A thorough scientific examination of the individual neurons would not predict the behavior of billions of them working together. • Emergent behavior • The system “software” is loaded via many years of training and education. Evidence of dynamic re-wiring and re-programming. • Networking: Neurons have massive connectivity to other neurons in their vicinity (LAN), although a complete neural network does not exist. However, chemical and hormonal networks are able to communicate to all neurons, and the brain can induce chemical signal outputs (WAN).

22. Advanced Emergent Behavior Management • So far, we have been mostly resisting, and combating emergent behavior. • Instead of building a dyke to keep out the waves, maybe we should be building a surfboard?

23. Advanced Emergent Behavior Management • Instead of avoiding emergent behavior (EB) with complex systems, take advantage of it! • Take advantage of the robust, adaptive and innovative natures of EB • Allow various system elements and the change agents to “get their way”, thus promoting new and innovative ideas • Charles Darwin was really on to something!

24. Advanced Emergent Behavior Management • Advanced EB Management Strategies: • Set up a controlled “ecosystem” that will allow the theory of evolution to play itself out (Darwin at warp speed). Make sure that successful “mutations” flourish quickly, and unsuccessful ones die faster. • But the “ecosystem” must accurately reflect the real world for the actual system usage. Nevertheless, it must stimulate change (speed up mutations). • Allow multiple teams (contractors) and change agents to compete and test their visions. May the fittest survive! • Make sure that there is a level playing field, and that all good ideas are recorded. • Keep in mind that stochastic elements of behavior, and intense networking are almost always required.

25. Advanced Emergent Behavior (from D. Fisher) • “Exploiting emergent behavior offers great potential for systems of systems, not only to overcome the problems of interoperation but also to achieve levels of adaptability, scalability, and cost-effectiveness not possible in traditional systems.” [11] David Fisher • “We need new software and systems engineering methods that manage emergent behavior and exploit emergent effects to offer the possibility of cost-effective and predictable solutions in systems of systems.” [11] • “A system-of-systems depends on emergent behaviors to achieve its purpose.” [11]

26. Questions?

27. Bibliography • “Managing Emergent Behavior in Distributed Control Systems”, H. Parunak and R. VanderBok, ISA-Tech 1997 • “Emergent (Mis)Behavior versus Complex Software Systems”, J. Mogul, 12/22/05 • “Engineering Emergent Protocols”, S. Bush and A. Kulkarni • “When Systems Engineering Fails - - Toward Complex Systems Engineering”, Y. Bar-Yam, New England Complex Systems Institute • “Emergent Processes”, T. Wiscombe • “AN EXPLORATION-BASED TAXONOMY FOR EMERGENT BEHAVIOR ANALYSIS IN SIMULATIONS”, Proceedings of the 2007 Winter Simulation Conference, R. Gore and P. Reynolds • Wikipedia • “Darwin Among the Machines: The Evolution of Global Intelligence”, George B. Dyson, Perseus Books Group, 1998. • WordiQ.com

28. Bibliography (cont.) • “Engineering Complex Systems”, D. Norman and M. Kuras, The MITRE Corporation • “An Emergent Perspective on Interoperation in Systems of Systems”, Fisher, David, (CMU/SEI-2006-TR-003), Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2006 • “Definition of the Sciences of Complex Systems”, S. Sheard, INCOSE INSIGHT, October 2006 • INCOSE Systems Engineering Handbook, Version 3