System of Systems Integration Cost Driver Research March 2004

1. Jeannine M. Siviy: SEPG 0545 System of Systems Integration Cost Driver Research March 2004

2. Agenda • Research Effort Background 1230-1300 • Initial Model and Hypotheses • Collaborators • Schedule Discussion 1300-1420 • Session Goals • Ground rules • Q&A • Wrap up Next Steps 1420-1430

3. Research Goals • To identify leading indicators of SoS cost and schedule • Gain insight into drivers of size, complexity, cost, and schedule • Gain insight into drivers of process efficiency and effectiveness • Output: • Analytical tools and methodologies to support Joint Capabilities investment decisions • Practical management methodologies to assist SoS/FoS implementers and operators

4. Research Status • Funded in FY04 as part of broad software research initiative • Current research partnership • Software Engineering Institute (SEI) • Dr. David Zubrow, SEI Software Engineering Measurement & Analysis (SEMA) • Mr. Bill Anderson, Integration of Software Intensive Systems (ISIS) • University of North Carolina (UNC) • Dr. Mary Maureen Brown, UNC School of Government • Defense Acquisition University (DAU) • Ms. Martha Spurlock, Director, Cost Analysis Curriculum • Mr. Robert Skertic, Software, Technical & Engineering Dept. • CAIG Study POC’s • Rob Flowe (703) 692-8052 robert.flowe@osd.mil • Tom Coonce (703) 697-3845 tom.coonce@osd.mil • Seeking collaboration opportunities, ideas, data!

5. Focus Group Goals • What key [issues, drivers, factors] differentiate the programmatic issues associated with SoS acquisition from single system development? • To what extent do the stakeholder, end-user, and developer relationships influence the technical architecture design and implementation? • To what extent is the strategic intent of the SoS effort accompanied by the necessary funding, operational, and personnel incentives to promote successful accomplishment?

6. Some General Precepts • “Interoperability” is a multi-dimensional aspect of system engineering. • Scope is far greater than simply interoperability of data – requires integration of enterprise rules • Scope includes degrees of operational, system, and technical coupling, ownership, … • Scope includes integration at the programmatic level • We can never anticipate fully the boundaries within which a given system will be expected to operate. • One man’s system-of-systems is another’s system. • There will always be new things to integrate into the system. • Integrating systems in a network can affect all other systems in the network in unintended ways.

7. More General Precepts • Size matters: • As integration in the small gets larger, new problems creep in: management, organizational. • As systems get more complex, interoperability issues increase. • As operators become more agile, systems must become dynamically reconfigurable. • Interoperability must be quantifiable to be achievable. • Interoperability must be sustainable and sustained.

8. A View of the Problem Space • Incomplete understanding of scope and nature of the engineering to be accomplished • There are weaknesses in translating high level concepts of operations into a set of system requirements that lead to interoperability. • Ongoing inertia toward separate programs executed independently • Technologies do not currently exist that permit quantification of interoperability.

9. We know quite a lot about constructing systems from components (over which we may have little or no control). Unplanned, unexpected, emergent behavior here… We know something about composing systems of systems from individual systems (over which we may have little or no control). System “B” Network “X” System “A” System “C” An Instance of the Problem We know very little about constructing an interoperable network of systems…the key distinction being that the network is unbounded (or very loosely bounded) and has no single controlling authority.

10. ProgramManagement SystemConstruction OperationalSystem Interoperability Activities Model Activities performed to manage the acquisition of a system. Activities performed to create and maintain a system. Focus is on architecture, standards, components. Activities performed to operate a system. Focus is on interactions with other systems and data distribution.

11. PM PM PM Programmatic Interoperability Construction Construction Construction Constructive System System System Interoperability Operational Interoperability SoSI Interoperability Model

12. Operational Interoperability • Fixing interoperability problems at the Operational level is key: Operational problems reduce the capabilities of the warfighter. • insufficient sharing of data among systems • sometimes operators make up for the deficiencies manually • sometimes unique interfaces obscure sharing that is possible • users uncover the needs • fixing problems in the field costs scarce resources • problems may not be found until time of greatest need

13. Constructive Interoperability • Developers may not know how their system is to interoperate with others; frequently they: • are given “just a piece” of the overall system of systems • SoS architectures are often insufficiently specified • have no model indicating the meaning of data (semantics) • are unaware of timing and sequencing of interactions • Technologies exist, but not clear how or when to use them. • Exceptions: technologies for real-time, high-speed applications and multilevel security are not sufficient • Information Exchange Requirements make sense in point-to-point communication, less so in a net-centric environment

14. Relationship among DoD AF Views

15. Programmatic Interoperability • Programmatic problems dominate constructive ones. • SoSs often lack an ultimate authority for overall system (of systems) engineering. • Requirements for interoperability either are not given or are vague. • Dependence on other systems can lead to local sub-optimization: no universal enforcement of policy. • Program offices are stove-piped, doing their own thing • even if they wanted to, it is hard to share information • “Unfunded mandates” • interoperability may not be sufficiently funded • interoperability is the first quality to be lost when schedule or money are tight • IAM is often not funded and therefore not developed

16. Characterizing Interoperability • Dimensions apply variously to three system aspects: • the whole system of systems • the components that make up the system of systems (“nodes”) • the relationships between the components (“arrows”) • Characteristics cover such dimensions as: • Evolution • Ownership • Technology availability • Communication • Planning and management • Usage patterns • Dynamism • Semantic consistency • Data management

17. Element Attributes Interface Attributes Product-Related Effort Drivers: e.g., size, & complexity Inherent Effort Enterprise Rules Part Of Cost Schedule Performance (Outcomes) Total Effort Drives Process-Related Effort Drivers: e.g., efficiency & effectiveness Part Of Process Attributes Induced Effort Environmental Attributes Concept Map of SoSI Attributes as Related to Acquisition Outcomes

18. Discussion

19. Ground Rules • Keep focused • Maintain momentum • Get closure on questions • Only one person speaks at a time • Be concise, get to the point • Stay on the topic and minimize diversions

20. Questions for Discussion • What key [issues, drivers, factors] differentiate the programmatic issues associated with SoS acquisition from single system development? • What differentiating activities in your experience are key to the success of SoS acquisition? • How do the differences impact cost, schedule, performance and functionality?

21. Questions for Discussion • To what extent do the stakeholder, end-user, and developer relationships influence the technical architecture design and implementation? Are the DoDAF views useful for translating operational capabilities into a roadmap for SoS integration? What impact do these relationships have on program cost, schedule, performance and functionality?

22. Questions for Discussion • To what extent is the strategic intent of the SoS effort accompanied by the necessary funding, operational, and personnel incentives to promote successful accomplishment? How do the requisite funding, operational, and personnel incentives impact program cost, schedule, performance and functionality?

23. Next Steps • Field Studies • Successful programs • Challenged programs • Ongoing Interviews with Subject Matter Experts • Points of Contact • Appreciate your comments on draft results and opportunity to continue collaboration • THANK – YOU!