Systems of Systems: Characteristics and Challenges

1. Systems of Systems: Characteristics and Challenges Barry Boehm, boehm@usc.edu USC Center for Systems & Software Engineering http://csse.usc.edu October 25, 2006

2. Overview • Definitions, Examples & Motivation • SoS Characteristics and Challenges • Conclusions

3. What is a “System-of-Systems”? • Very large systems developed by creating a framework or architecture to integrate component systems • SoS component systems independently developed and managed • New or existing systems • Some outsourced • Some externally evolving • Have their own purpose • Can dynamically come and go from SoS • SoS exhibits emergent behavior not otherwise achievable by component systems • SoS activities often planned and coordinated by a Lead System Integrator (LSI) • Typical domains • Business: Enterprise-wide and cross-enterprise integration to support core business enterprise operations across functional and geographical areas • Military: Dynamic communications infrastructure to support operations in a constantly changing, sometimes adversarial, environment

4. What Does an SOS Look Like: Future Combat Systems

6. The Need for Net-Centric Systems of Systems (NCSOS) • Lack of integration among stove-piped systems causes • Unacceptable delays in service • Uncoordinated and conflicting plans • Ineffective or dangerous decisions • Inability to cope with fast-moving events • Increasing NCSOS benefits • See first; understand first; act first • Network-centric operations coordination • Transformation of business/mission potential • Interoperability via Integrated Enterprise Architectures

7. Overview • Definitions, Examples & Motivation • SoS Characteristics and Challenges • Conclusions

8. SoS Characteristics and Challenges - I • Complexity: Size and number of organizations, interfaces, suppliers, coordination groups • Scalability of processes, methods, and tools • Dynamism: Number of changes/month, average time to process change • Rapid change impact analysis, change synthesis, negotiation, development, validation, implementation • Emergence: requirements not pre-specifiable • Build-to-spec processes, contracts infeasible • C2ISR a better metaphor for SoS acquisition than purchasing-agent • Command, control, intelligence, surveillance, reconnaissance

9. Complexity of SoS Solution Spaces • Size: 10-100 MLOC • Number of external interfaces: 30-300 • Number of “Coopetitive” suppliers: 20-200 • Even more separate work locations • Depth of supplier hierarchy: 6-12 levels • Number of coordination groups: 20-200 • Reviews, changes, risks, requirements, architecture, standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,… • Key personnel spend 60 hours/week in meetings • Unprecedentedness • Emergence • Rapid change Necessarily software-intensive…

10. Average Change Processing Time: 2 SoSs • Average workdays to process changes

11. Observe new/updated objectives, constraints, alternatives • Usage monitoring • Competition, technology, marketplace ISR • Orient with respect to stakeholders priorities, feasibility, risks • Risk/Opportunity analysis • Business case/mission analysis • Prototypes, models, simulations Operate as current system Accept new system • Decide on next-cycle capabilities, architecture upgrades, plans • Stable specifications, COTS upgrades • Development, integration, V&V, risk management plans • Feasibility rationale • Act on plans, specifications • Keep development stabilized • Change impact analysis, preparation for next cycle (mini-OODA loop) Life Cycle Architecture Milestone for Cycle Acquisition C2ISR Via Spiral OODA Loops Example: ARPANet/Internet Spiral

12. SoS Characteristics and Challenges - II • COTS/Reuse/Legacy diversity • Many sources of incompatibility, changes • COTS: average 8-10mo/release; unsupported after 3 releases • Multiple missions and stakeholders to support • Increment and change content requires negotiation • Independently evolving systems • Often with “coopetitive” suppliers, interoperators • More time needed for systems definition • Before and after source selection • More time needed for teambuilding, partner coordination, supplier management, change management , integration and test

13. 10000 KSLOC Sweet Spot 100 KSLOC Sweet Spot Drivers: Rapid Change: leftward High Assurance: rightward 10 KSLOC How much Architecting is Enough: COCOMO II Analysis Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution Added Schedule Devoted to Rework (COCOMO II RESL factor) Total % Added Schedule

14. COSOSIMO: I&T Sub-Model • Size Drivers • # SoS interface protocols • # SoS scenarios • # unique component systems SoS Integration and Testing LSI I&T Effort • Cost Drivers • Requirements understanding • Architecture maturity • Level of service requirements • SoS team capability • Maturity of LSI processes • Tool support • Cost/schedule compatibility • SoS risk resolution • Component system maturity and stability • Component system readiness

15. Conclusions • Individual SoS attributes are highly challenging • Complexity, dynamism, emergence, uncontrollables, stakeholder diversity • Their combinations are even more challenging • Acquisition management and negotiation skills are at least as important as systems engineering skills • C2ISR a better metaphor for SoS acquisition than purchasing-agent • More time needed for systems definition • Before and after source selection

16. Backup Charts

17. Platform N • • • Platform 1 Infra C4ISR Width DOTMLPF 1.0 2.0 3.0 4.0 5.0 … Length 2008 2010 2012 2014 2016 Command and Control Situation Assessment Info Fusion Sensor Data Management Sensor Data Integration Sensors Sensor Components : Depth Complexity of Solution Spaces- Breadth, Depth, and Length Legend: DOTMLPF Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities C4ISR Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance

18. Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May 2004 • Acquisition management and staffing • Requirements/architecture feasibility • Achievable software schedules • Supplier integration • Adaptation to rapid change • Quality factor achievability and tradeoffs • Product integration and electronic upgrade • Software COTS and reuse feasibility • External interoperability • Technology readiness

19. $100M Arch. A: Custom many cache processors $50M Arch. B: Modified Client-Server Available budget Original Spec After Prototyping 5 3 1 2 4 Response Time (sec) Effect of Unvalidated Requirements-15 Month Architecture Rework Delay

20. Effect of Unvalidated Software Schedules • Original goal: 18,000 KSLOC in 7 years • Initial COCOMO II, SEER runs showed infeasibility • Estimated development schedule in months for closely coupled SW with size measured in equivalent KSLOC (thousands of source lines of code): • Months =~ 5 * 3√KSLOC • Solution approach: architect for decoupled parallel development; • Schedule As Independent Variable (SAIV) process

21. The SAIV* Process Model • Shared vision and expectations management • Feature prioritization • Schedule range estimation and core-capability determination • Top-priority features achievable within fixed schedule with 90% confidence • Architecting for ease of adding or dropping borderline-priority features • And for accommodating past-IOC directions of growth • Incremental development • Core capability as increment 1 • Change and progress monitoring and control • Add or drop borderline-priority features to meet schedule *Schedule As Independent Variable; Feature set as dependent variable. Also works for cost, schedule/cost/quality as independent variable.

22. Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May 2004 • Acquisition management and staffing • Requirements/architecture feasibility • Achievable software schedules • Supplier integration • Adaptation to rapid change • Quality factor achievability and tradeoffs • Product integration and electronic upgrade • Software COTS and reuse feasibility • External interoperability • Technology readiness

23. COTS UpgradeSynchronization and Obsolescence • Many subcontractors means an increasing number of evolving COTS interfaces • Aggressively-bid subcontracts can deliver obsolete COTS • New COTS released every 8-9 months (GSAW) • COTS unsupported after 3 releases (GSAW) • An actual delivery: 120 COTS; 46% unsupported • Emphasize COTS interoperability in source selection process • Develop contract/subcontract provisions/incentives to ensure • Consistency and interoperability across contractor and subcontractor-delivered COTS software products • Such products are recent-release versions • Develop a management tracking scheme for all COTS software products in all NCSOS software elements • Develop a strategy for synchronizing COTS upgrades

24. Emerging Scalable Spiral Process- For 21st century systems engineering and acquisition • The C4ISR Metaphor for NCSOS Acquisition • Role of OODA loops • Example: Internet Spiral • Example: FCS Win Win Spiral Model • Risk-Driven Scalable Spiral Model • Increment view • Life-cycle view • Role of anchor point milestones • Acquisition management implications • People management implications

25. Risk-Driven Scalable Spiral Model:Increment View

26. Risk-Driven Scalable Spiral Model:Increment View

27. System LCA System, DI1 LCA DI2 B/L LCA System Inception System Elaboration Agile DI2 (OO&D) Rebaselining Plan-Driven DI1 Construction (A) DI1 V&V DI2 LCA Plan-Driven DI2 Construction (A) DI2 V&V Changes Risk-Driven Scalable Spiral Model:Life Cycle View LCA: Life Cycle Architecture IOC: Initial Operational Capability OO&D: Observe, Orient and Decide V&V: Verification and Validation DI: Development Increment B/L: Baselined

28. System LCA System, DI1 LCA DI2 B/L LCA DI3 B/L LCA DI4 B/L LCA System Inception System Elaboration Agile DI2 (OO&D) Rebaselining Update Update DI1 IOC Plan-Driven DI1 Construction (A) DI1 Trans’n DI1 V&V DI2 LCA Agile DI3 (OO&D) Rebaselining Update DI2 IOC Plan-Driven DI2 Construction (A) DI2 Trans’n DI2 V&V DI3 LCA Agile DI4 (OO&D) Rebaselining DI3 IOC LCA: Life Cycle Architecture IOC: Initial Operational Capability OO&D: Observe, Orient and Decide V&V: Verification and Validation DI: Development Increment B/L: Baselined Plan-Driven DI3 Construction (A) DI3 Trans’n DI3 Usage . . . DI3 V&V DI4 LCA . . . DI1 Usage DI2 Usage Changes Changes Changes Risk-Driven Scalable Spiral Model:Life Cycle View

29. LCO (MS A) and LCA (MS B) Anchor Points Pass/Fail Criteria • A system built to the given architecture will • Support the operational concept • Satisfy the requirements • Be faithful to the prototype(s) • Be buildable within the budgets and schedules in the plan • Show a viable business case • Establish key stakeholders’ commitment to proceed LCO: True for at least one architecture LCA: True for the specific life cycle architecture; All major risks resolved or covered by a risk management plan

30. Spiral Feasibility Rationale Deliverable • LCO, LCA reviews not just UML/PowerPoint charts • Need to show evidence of product and process feasibility • Evidence provided by prototypes, production code, benchmarks, models, simulations, analysis • Sizing and cost/schedule model results for process feasibility • Evidence provided in advance to LCO/LCA review team • Key stakeholders, specialty experts • Lack of evidence risks destabilizing the process • Needs coverage by viable risk mitigation plan • Key new progress metric • Feasibility evidence progress vs. plans

31. B/L Baselined C4ISR Command, Control, Communications, Computers, Intelligence, Surveillance, and Reconnaissance CCD Core Capability Drive-Through COTS Commercial Off the Shelf CRACK Collaborative, Representative, Authorized, Committed, Knowledgeable DI Development Increment DODAF Department of Defense Architectural Framework DOTMLPF Doctrine, Organization, Training, Materiel, Leadership, Personnel, Facilities ERP Enterprise Resource Planning FEAF Federal Enterprise Architectural Framework GSAW Ground System Architectures Workshop IESG Internet Engineering Steering Group IETF Internet Engineering Task Force IKIWISI I’ll Know It When I See It IOC Initial Operational Capability IPT Integrated Product Team IRR Inception Readiness Review LCA Life Cycle Architecture LCO Life Cycle Objectives LSI Lead System Integrator MLOC Million Lines of Code MS Milestone NCSOS Net-Centric System of Systems OO&D Observe, Orient, and Decide OODA Observe, Orient, Decide, Act PM Person-Month/Program Manager PRR Product Release Review SAIV Schedule As Independent Variable SE System Engineering SoS System of Systems SOW Statement of Work SW Software Sys Engr System Engineering V&V Verification and Validation WMI War-fighter machine interface Acronym Definitions