Engineering Large Evolving Systems

1. Engineering Large Evolving Systems Ben Potter Department of Informatics and Sensors, Cranfield University

2. Contents • The case study – Network Enabled Capability • Challenges for engineering • Discussion

3. What is NEC? • The attainment of a Network Enabled Capability in UK Defence • Sometimes also referred to as Capability Enhanced by Networking – not as snappy but perhaps more descriptive! • UK MOD Joint Service Publication JSP 777 says: • ‘The achievement of military effect will, in future, be significantly enhanced through the networking of existing and future military capabilities, under the banner of NEC’

4. What is NEC ? Working definition (from MOD website): "Linking sensors, decision makers and weapon systems so that information can be translated into synchronised and overwhelming military effect at optimum tempo"

5. What is NEC ? “NEC is about the coherent integration of sensors, decision-makers and weapon systems along with support capabilities”

6. A Navy view?

7. An Army view?

8. An aside • 1976 Networkshop…

9. Some questions: • Given (say) a 12 year horizon for NEC: • Who can describe the military threats to be met? • Who can imagine the technologies to be used? • How can ‘traditional’ systems engineering help, given that: • 1 above suggests we can’t write the requirements • 2 above suggests we can’t describe a solution which will not be obsolete • First – a definition: Engineered Large Yet Evolving Systems – ELYES. NEC is an ELYES. • This leads us to appeal to the origins of systems engineering for help – or, at least – guidance

10. What is a ‘System?’

11. System as a set of interacting parts “A system is an open set of complementary, interacting parts with properties, capabilities, and behaviours emerging both from the parts and from their interactions.” Hitchins, Advanced Systems Thinking, Engineering and Management, 2003

12. System as a way of conceptualising “… the concept of ‘system’ is used not to refer to things in the world but to a particular way of organising our thoughts about the world. [..] we consider the notion of ‘system’ as an organising concept …” Flood and Jackson, Creative Problem Solving – Total Systems Intervention, 2004

13. Different ‘organising concepts’ Engineer Chancellor of Exchequer Cyclist Family Person

14. The System Boundary “… the scientist has to have a way of thinking about the environment of a system that is richer and more subtle than a mere looking at for boundaries. He does this by noting that, when we say that something lies ‘outside’ the system, we mean that system can do relatively little about its characteristics or its behavior. Environment, in effect, makes up the things and people that are ‘fixed’ or ‘given’, from the system’s point of view. …” Churchman, The Systems Approach, 1968

15. A system’s emergent properties “Emergence is the phenomenon of properties, capabilities and behaviours evident in the whole system that are not exclusively ascribable to any of its parts.” Hitchins, Advanced Systems Thinking, Engineering and Management, 2003

16. Summary: a system is … • A organising concept – a way of structuring a concept or problem or a thing from a particular perspective. • A system is made up of a set of interacting parts • The system has properties and behaviour that emerge from the interaction of the properties and behaviours of the individual parts. • The system has a boundary defined in terms of ‘things that affect it but that it cannot affect.’ – but the boundary can be broad and permeable.

17. How do we analyse ‘Systems’?

18. How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969

19. How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969

20. How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’.The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969

21. How we analyse systems “… This is the basic principle of ‘classical’ science, which can be circumscribed in different ways: resolution into isolable casual trains, seeking for ‘atomic’ units in the various fields of science, etc. The progress of science has shown that these principles of classical science – first encountered by Galileo and Descartes – are highly successful in a wide range of phenomena. Application of the analytical procedure depends on two conditions. The first is that interactions between ‘parts’ be non-existent or weak enough to be neglected for certain research purposes. Only under this condition, can the parts be ‘worked out’, actually, logically and mathematically, and then be ‘put together’. The second condition is that the relations describing the behaviour of the parts be linear; only then is the condition of summativity given …” Bertalanffy, General Systems Theory, 1969

22. How we analyse systems “… the method of classical science was most appropriate for phenomena that either can be resolved into isolated causal chains, or are the statistical outcome of an ‘infinite’ number of chance processes, as is true of statistical mechanics, the second principle of thermodynamics and all the laws deriving from it. The classical modes of thinking, however, fail in the case of interaction of a large but limited number of elements of processes. Here those problems arise which are circumscribed by such notions as wholeness, organisation and the like, and which demand new ways of thinking.” Bertalanffy, General Systems Theory, 1969

23. How we analyse systems “… the method of classical science was most appropriate for phenomena that either can be resolved into isolated causal chains, orare the statistical outcome of an ‘infinite’ number of chance processes, as is true of statistical mechanics, the second principle of thermodynamics and all the laws deriving from it. The classical modes of thinking, however, fail in the case of interaction of a large but limited number of elements of processes. Here those problems arise which are circumscribed by such notions as wholeness, organisation and the like, and which demand new ways of thinking.” Bertalanffy, General Systems Theory, 1969

24. How we analyse systems “… the method of classical science was most appropriate for phenomena that either can be resolved into isolated causal chains, or are the statistical outcome of an ‘infinite’ number of chance processes, as is true of statistical mechanics, the second principle of thermodynamics and all the laws deriving from it. The classical modes of thinking, however, fail in the case of interaction of a large but limited number of elements of processes. Here those problems arise which are circumscribed by such notions as wholeness, organisation and the like, and which demand new ways of thinking.” Bertalanffy, General Systems Theory, 1969

25. Statistics and dynamics relating to very large numbers of well defined nodes and couplings. Nodes that have ‘adaptive’, non-trivial couplings. Decoupled or very well defined couplings between relatively few nodes. ‘System space’ Increasing number of ‘nodes’ Increasing intricacy of the ‘couplings’

26. Coping strategies Increasing number of ‘nodes’ Tendency to try to inappropriately extend ‘comfort zone’ techniques Tendency to treat ‘complex’ systems within ‘comfort zones’ Increasing intricacy of the ‘couplings’

27. Weather Climate Flock of birds Termite hill Space shuttle Aircraft Car Example Systems Gas in a container Increasing number of ‘nodes’ Organisations Teams Increasing intricacy of the ‘couplings’

28. Gas in a container Car Driver Engine Drive Train Fuel system Gas atoms and molecules Detailed example systems Emergent behaviour: ride comfort, acceleration… Emergent behaviour: pressure… … and can be determined through statistical and probabilistic analysis … and is very determinant

29. Military Capability Command Prepare Inform Project Operate Sustain Protect Detailed example ELYES Emergent behaviour: effects… … and cannot be determined to any degree of certainty

30. Weather Climate Flock of birds Organisations Termite hill Space shuttle Aircraft ELYES System Car Example Systems Gas in a container Increasing number of ‘nodes’ Organisations Teams Increasing intricacy of the ‘couplings’

31. Architecting an ELYES • ELYES Architecture is concerned with optimisation at the global level, sometimes necessarily at the expense of local considerations • In pursuit of this goal, the ELYES Architect may be tempted to over-simplify and • Adopt a one-size-fits-all approach • See more homogeneity in the ELYES than there really is • Assume linear, predictable systems • Focus on static, structural elements, while ignoring dynamic interactions • While this response is understandable, is it reasonable?

32. The problem • Changing environments require systems/ELYESs to change: • Either to reflect the changes in the environment or anticipate them. • The unpredictable nature of the environment includes: • Dynamic entities (threats and contributors) within the environment. • New entities that are constantly evolving and old ones disappearing. • Relationships between the entities that are constantly changing. • The overall behaviour exhibited by inter-related entities are emergent. • The characteristics of the system/ELYES have to be at least as complex as its environment.

33. Thesis / Premise • Traditionally we design systems by identifying the ‘static’ customer need, using a top-down, reductionist approach. This single mode of analysis struggles to cope with: • Dynamic interactions within the system • Non-functional requirements (e.g. agility and flexibility) • Complex emergent properties and behaviours • We argue that for ELYESs to operate it requires appropriate developmental and operational environments in which traditional systems can be developed and used.

34. ELYES Systems System is reproducible No two instantiations are alike System is realised to meet a single, pre-conceived need No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System has well-defined boundaries and behaviour. The ELYES has ambiguous boundaries and unpredictable behaviour. Development always ends for each instance of system realisation ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES. Unwanted possibilities and unknowns are removed before use of the system Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Systems and ELYESs

35. Systems: Implications for development Systems Implications for Development Once the design is fixed the production can be ‘mechanized’. System is reproducible The design can be fixed, the ‘need’ does not change once it has been agreed. There is an agreed ‘product’. System is realized to meet a single, pre-conceived need Development and production can be decomposed into sub-products with well defined behaviours, which can then be composed to form the whole. System has well-defined boundaries and behaviour. Development always ends for each instance of system realization No further development once the product is handed over to the user. External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The user cannot change the product. Unwanted possibilities and unknowns are removed before use of the system The behaviour of the delivered product is bounded and known. Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The product is not delivered until the user knows how to use it. Design, production and use can be treated as totally separate phases with mechanised processes and well defined boundaries and products. Very clear distinction between development/design, production and use.

36. ELYESs: Implications for development ELYESs Implications for Development The ‘design’ cannot be fixed and the production cannot be ‘mechanized’. No two instantiations are alike No single, or set of needs can be pre-conceived for the System. The system has to continually evolve to meet the changing need. The need for the ELYES is continually changing. The ELYES can only evolve through use. Not all sub-units can be identified and some may not be controllable. Achieving the desired behaviour is about creating the appropriate environment and influencing sub-units. System has ambiguous boundaries and unpredictable behaviour. An appropriate environment has to be developed that lets the system evolve through use. System development never ends – system evolves The user has freedom to re-integrate as required. The ELYES must be designed to be ‘modular’ with configuration ‘rules’. System is self-integrating and re-integrating in order to meet the changing need. The ELYES is never completed and the role of development and production is to provide the user with the means for evolution. New possibilities are constantly assessed for utility and feasibility in the evolution of the system The ELYES is heterogeneous and relies on conflicts and collaborations to evolve. These must not be ‘designed out’. System depends on both internal cooperation and internal competition to stimulate its evolution. The development, production and use lifecycle must be adaptable and not prescriptive. Development, production and use all run in parallel.

37. System acquisition Require-ments Programme

38. ELYES Acquisition ? ? ? Principles ? Programme

39. It’s about setting the right environment NOT designing top down. Implications for engineeringan ELYES

40. Designing the ELYES Operational Need ELYES Architecture ELYES Design Development Environment Cap 1 Cap 2 Cap n Cap 3 Kit Kit Kit Kit Operational Environment

41. Evolving the ELYES ELYES Architecture ‘Big Picture’ Partition Rules Operational Need Development Environment Orchestration Rules Cap 1 Cap n Kit Kit Kit Orchestration Rules Operational Environment

42. Implications for engineeringan ELYES • Capability development is enabled by providing the appropriate development environment. • Number of and definition of capability building blocks • Cannot be predetermined • Will evolve independently • Will have overlapping functionality with one another. • Redundancy, i.e. variety, in the system is desirable • Operational use of the capability is defined by orchestration rules which will encourage appropriate ‘usage’ philosophy. • The orchestration rules are defined during development and will evolve. • Evolution during operational use must drive development.

43. Discussion

44. ELYES Systems System is reproducible No two instantiations are alike No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System is realized to meet a single, pre-conceived need The ELYES has ambiguous boundaries and unpredictable behaviour. System has well-defined boundaries and behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYESr evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel. Ability to do stuff Example: A Bridge Target Specification Time

45. ELYES No two instantiations are alike No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. The ELYES has ambiguous boundaries and unpredictable behaviour. ELYES development never ends – the ELYES evolves The ELYES is self-integrating and re-integrating in order to meet the changing need. New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES. The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Development, production and use all run in parallel. Ability to do stuff Example: Software Target Specification Time Systems System is reproducible System is realized to meet a single, pre-conceived need System has well-defined boundaries and behaviour. Development always ends for each instance of system realization External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. Unwanted possibilities and unknowns are removed before use of the system Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed Very clear distinction between development/design, production and use.

46. ELYES No two instantiations are alike No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. The ELYES has ambiguous boundaries and unpredictable behaviour. ELYES development never ends – the ELYES evolves The ELYES is self-integrating and re-integrating in order to meet the changing need. New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Development, production and use all run in parallel. Ability to do stuff Example: Space Race Target Specification Sub-Target Sub-Target Sub-Target Time Systems System is reproducible System is realized to meet a single, pre-conceived need System has well-defined boundaries and behaviour. Development always ends for each instance of system realization External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. Unwanted possibilities and unknowns are removed before use of the system Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed Very clear distinction between development/design, production and use.

47. Ability to do stuff Example:My kitchen Specification(Tolerance) Time ELYES Systems System is reproducible No two instantiations are alike No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. System is realized to meet a single, pre-conceived need The ELYES has ambiguous boundaries and unpredictable behaviour. System has well-defined boundaries and behaviour. Development always ends for each instance of system realization ELYES development never ends – the ELYES evolves External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. The ELYES is self-integrating and re-integrating in order to meet the changing need. Unwanted possibilities and unknowns are removed before use of the system New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Very clear distinction between development/design, production and use. Development, production and use all run in parallel.

48. ELYES No two instantiat6ions are alike No single, or set of needs can be pre-conceived for the ELYES. The ELYES has to continually evolve to meet the changing need. The ELYES has ambiguous boundaries and unpredictable behaviour. ELYES development never ends – the ELYES evolves The ELYES is self-integrating and re-integrating in order to meet the changing need. New possibilities are constantly assessed for utility and feasibility in the evolution of the ELYES The ELYES depends on both internal cooperation and internal competition to stimulate its evolution. Development, production and use all run in parallel. Ability to do stuff Example:NEC Specification(Range of uses) Time Systems System is reproducible System is realized to meet a single, pre-conceived need System has well-defined boundaries and behaviour. Development always ends for each instance of system realization External agents (i.e. not the users) integrate system into its final form, to meet a prescribed need. Unwanted possibilities and unknowns are removed before use of the system Development ends when unwanted sources of internal friction (competition for resources, differing interpretations of the same inputs, etc.) are removed Very clear distinction between development/design, production and use.

49. Summary: Lifecycles Software Bridge Space Race Kitchen NEC