NDIA SoS SE Committee Topics of Interest

1. NDIA SoS SE Committee Topics of Interest May 25, 2009

2. Overview • New NDIA SoS SE Committee formed with kickoff meeting in April • Committee members submitted emails identifying SoS issues which ‘keep them away at night” • These ‘sleepless night’ issues were aggregated • Members were asked to identify their top 5 issues and the questions they would like to see addressed with respect to these issues • This presentation summarized the results

3. 23 Sleepless Night Issues 1 Lessons learned from past and on-going SoS programs 2 Perspective of the SoS manager for SoS SE 3 Defining SoS relationship in terms of an 'SoS Signature' 4 Relationship among SE approaches for systems, SoS, FoS 5 Difference in SoS definitions 6 Developing validated SoS policy 7 Relationship between SoS and acquisition process 8 Addressing SoS from perspective of a new system acquisition 9 Role of requirements in SoS 10 SoS architecture 11 SoS Integration 12 SoS and Complexity 13 Role of optimization in SoS SE 14 Defining and measuring SoS performance 15 SoS test and evaluation 16 SoS SE competencies and training 17 SoS governance 18 Constructive approaches to addressing politics in an SoS 19 SoS business models 20 Relationship between SoS and net-centric services 21 Science based theory to support SE of systems and SoS 22 SoS modeling and simulation 23 SoS SE Analyses of CBRN

4. Spreadsheet Soliciting Inputs • Spreadsheet was emailed to ~60 people indicating an interest in the SoS SE Committee • Request for top 5 topics and questions of interest for these topics • Received inputs from 14 members

5. Results • 1 Issue 8 (of 12) expressed interest • 1 Issues 7 (of 12) expressed interest • 1 Issue 6 (of 12) expressed interest • 2 Issues 5 (of 12) expressed interest • 1 Issues 4 (of 12) expressed interest • 2 Issues 3 (of 12) expressed interest • 3 Issues 2 (of 12) expressed interest • 10 Issues 1 (of 12) expressed interest • 4 Issues 0 (of 12) expressed interest

6. Top Issues • SoS Test and Evaluation (8 of 12) • Relationship between SoS and acquisition process (7 of 12) • Lessons Learned from past and on-going SoS programs (6 of 12) • Defining and measuring SoS performance (5 of 12)[related questions were posed under SoS Test and Evaluation] • SoS modeling and simulation (5 of 12)[Already a topic of NDIA M&S Committee] • SoS architecture (4 of 12)

7. Top Issue: SoS Test and Evaluation (8 of 12) • How to evaluate SoS Performance metrics in T&E • Traditional techniques for T&E of individual systems do not always scale well to SoS. • What are some new constructs for SoS T&E? • How should model and simulation development progress, in step with SoS development, from concept to test and evaluation? • How to plan & leverage T&E throughout out SoS L/C? • How to assess implications of changes/new capabilities in constituent systems on SoS T&E? • Does this premise extend the underlying effectiveness, suitability, and survivability (ESS) test requirements at a system level? • What is the relationship of SoS test and evaluation requirements to the conceptual requirements of AEC's MBT&E process? • Should and how are SoS test and evaluation requirements defined in the TEMP and SEP? • How do you define success for a SoS? In operational measures such as mission success or by other more technical measures? • What are the means to determine the weak links and strong links within the SoS and to quantify the attributes that make sub-component strong?"

8. Relationship between SoS and the acquisition process (4 of 12) • What are the barriers/challenges or potential incentives to removing same when acquiring SoS (in non-Directed SoS)? • What specific milestones, decisions, or products is the SoS approach expected to inform along the JCIDS and DAS processes? • Are there any specific and unique products that the SoS approach is expected to produce and what are the relationships of these products to acquisition lifecycle products? • How does the SoS support the acquisition process? • What are the differences between acquiring a System and a SoS?

9. Lessons Learned from past and on-going SoS programs (6 of 12) • What is the recommended approach and what are the basic steps to design large integrated SoSs where the systems are independently and separately developed? • What is the recommended approach and what are the basic steps to design large integrated network centric SoSs where the systems are independently and separately developed? • What are the recommended suite of methods and tools that can be used to design large SoSs including integrated network SoSs? • What are best practices and lessons learned for IT-intensive SoS acqusitions? • As the Lead Systems Integrator for the FCS Program, how did Boeing organize for SoS management? What lessons did Boeing learn (5 "up" lessons and 5 "down" lessons)? • What are the best practices to surface issues with respect to achieving (software) interoperabilty? At which parts of life cycle are they most effective?

10. Next Steps • Note small number of respondents and consider ways to get better input • Discuss with committee at next meeting • Are the top three an ok starting point? Or do we need to solicit more input? • Begin addressing top issue: SoS and T&E • Circulate our SoS T&E paper to committee members now • Paper raises almost all of the questions note in the input and provides one perspective to focus discussion • Solicit comments on it by 3 June • Make this a discussion topic at next meeting • Invite T&E committee to join us? • Include AFEI SOA outbrief at next meeting • Partnership with AFEI (plus input from NSA) • Topic of IT intensive systems and net-centric systems raised in several of the lessons learned topic questions