Stevens-USC SERC: A DoD University Affiliated Research Center (UARC)

1. Stevens-USC SERC:A DoD University Affiliated Research Center (UARC) Barry Boehm, USC USC-CSSE Annual Research Review March 16, 2009

2. Outline • Nature of UARCs • SERC Overview • SERC organization and vision • SERC research strategy • Initial tasks • SysE Effectiveness Measures Assessment (EM) • SysE Methods, Processes, Tools Evaluation (MPT) • Workshop objectives and approach

3. What is a UARC? • University Affiliated Research Centers are “not-for-profit, private sector organizations affiliated with, or part of, universities or colleges that maintain essential research, development and engineering capabilities needed by sponsoring DoD components.” • They maintain long-term, strategic relationships with sponsoring DoD components in specific core areas and operate in the public interest, free from real or perceived conflicts of interest. • UARCs are financed through long-term, non-competitive contracts awarded by sponsoring DoD components for specific core work.

4. Several Existing UARCs • Johns Hopkins University APL; 4,300 people; annual funding $680M • UC Santa Cruz with NASA Ames – information technology, biotechnology, nanotechnology, computer science, aerospace operations, astrobiology, and fundamental biology. Has a Systems Teaching Institute with San Jose State University and UCSC to teach through hands on experience on research projects. • Penn State University Applied Research Laboratory for the Navy with focus on undersea missions and related areas; strategic partner with NAVSEA and ONR; established 1945; has >1000 faculty and staff • University of Washington APL – acoustic and oceanographic studies ; established in 1943 • UC Santa Barbara Institute for Collaborative Biotechnology – Army Research Office; partnered with MIT and Cal Tech – focus on biologically-derived and biologically-inspired materials, sensors, and information processing … • University of Texas UARC started in 1945 focuses on sonar, acoustics, software system research, satellite geodesy, active sonar, …; now has 600 people on staff • USC Institute for Creative Technology – US Army; focus on virtual reality multimedia applications for training, C4ISR

5. Auburn University Air Force Institute of Technology Carnegie Mellon University Fraunhofer Center at UMD Massachusetts Institute of Technology Missouri University of Science and Technology (S&T) Naval Postgraduate School SERC Organization Lead organizations Members • Pennsylvania State University • Southern Methodist University • Texas A&M University • Texas Tech University • University of Alabama in Huntsville • University of California at San Diego • University of Maryland • University of Massachusetts • University of Virginia • Wayne State University As the DoD Systems Engineering Research-University Affiliated Research Center, SERC will be responsible for systems engineering research that supports the development, integration, testing and sustainability of complex defense systems, enterprises and services. Its members are located in 11 states, near many DoD facilities and all DAU campuses.

6. SERC Organization - II Dr. Dinesh Verma Executive Director Dr. Art Pyster Deputy Executive Director Julie Norris (acting) Director of Operations Dr. Barry Boehm Director of Research Pool of more than 140 Senior Researchers and hundreds of research faculty and graduate students from across members Stevens’ School of Systems and Enterprises will host the SERC at Stevens’ Hoboken, NJ, campus. Stevens’ faculty engagement will be complemented by a critical mass of systems engineering faculty at USC. A fundamental tenet of SERC is its virtual nature – each of its 18 members will be a nexus of research activities. All research projects will be staffed by the best available researchers and graduate students from across the members.

7. Rough Financial Model • Minimum of $2M/year • Can add more funded tasks within contract • First year: two specified tasks • 1. SysE Effectiveness Measurement: EM (USC lead; $500K) • 2. SysE Methods, Processes, Tools Assessment : MPT (Stevens lead; $350K) • Further tasks proposals being submitted • In response to SERC Research Strategy Thrusts • Other Govt. sponsors can sole-source through UARC • Procedures being worked out • Industry, FFRDC collaboration modes being worked out

8. SERC Vision and Perspective DoD and IC systems achieving mission outcomes – enabled by research leading to transformational SE methods, processes, and tools. Vision Perspective • The SERC will be the primary engine for defense and intelligence community SE basic research. In doing so, the SERC will: • Transform SE practice throughout the DoD and IC communities by creating innovative methods, processes, and tools that address critical challenges to meeting mission outcomes (what), • Become the catalyst for community growth among SE researchers by enabling collaboration among many SE research organizations (who), • Accelerate SE competency development through rapid transfer of its research to educators and practitioners (how).

9. SERC Research Strategy Thrusts • Enterprise Responsiveness • Systems Science and Complexity • Human Capital • Program and Systems Engineering Integration • Life Cycle Systems Engineering Processes

10. 1. Enterprise Responsiveness Thrust • Collaboration. Research innovative collaborative technologies to dramatically improve systems engineering is performed, especially for teams that are geographically, culturally, and linguistically diverse. • Modeling and Simulation. Research how to more rapidly and easily develop modeling and simulation tools that help validate concepts of operation, architectures, and other key systems engineering artifacts. • Resilient Program. Research how to architect and implement a program that is resilient in the face of a wide range of potentially harmful changes, such as budget cuts, requirements turbulence, failure of needed technology maturation, and program redirection

11. 1. Enterprise Responsiveness Thrust - II • Resilient System. Research how to architect and implement a system that is resilient in the face of a wide range of threats, both cyber and physical. • Producibility. Research how to dramatically decrease the amount of effort and schedule required for implementing new software-intensive systems. • Parsimony. Research how to reduce the amount and size of SE artifacts, e.g., determining minimum-essential architectural views number, detail, rigor, and capture time. • Strategic Assessment. Research improved process models and decision criteria for better SE support of strategic and enterprise decisions.

12. 2. Systems Science and Complexity • Composition. Research how to determine key properties of a system (assurance, scalability, availability, producibility, interoperability, lethality, resilience, …) when (a) the properties of its subsystems are known, (b) the system architecture that links those subsystems is known, and (c) the concept of operations is known. • Emergence.Research how to manage emergence in requirements, technology, and system usage in a way that is beneficial or benign rather than disruptive to development and acquisition programs; and how to anticipate and mitigate emergent behaviors. • System Conceptualization. Research approaches to better visualize and define system concepts that enable collaboration among multiple stakeholders.

13. 2. Systems Science and Complexity - II • Holistic SE. Investigate systems thinking methodology and its application to DoD enterprise and capabilities planning. • Validation. Research how to improve the early and continual validation of complex, expensive systems. • Transformation. Take systems engineering and systems thinking MPTs that are applied to critical contemporary issues outside the DoD domain (such as creating a more cost effective health care system for the U.S. general population) and transform them so as to be directly applicable to problems in the DoD domain.

14. 3. Human Capital • Collaboration and Education. Extends Enterprise Collaboration thrust element to include education. • Acceleration.Research how to accelerate the growth of the systems engineering workforce for both DoD and its contractors. • Dispersion. Research how to rapidly and effectively grow SE competencies in project managers, contracting officers, and other acquisition personnel besides SEs for whom SE competency could enhance their performance. • Staffing. Research how to staff a program with the right mix of systems engineers as a function of program type, size, complexity, risk profile, and other system and system acquisition characteristics.

15. 4. Program and Systems Engineering Integration • SE Effectiveness. Research SE program execution performance, return on investment, and metrics to demonstrate and improve value of SE. • Value. Determine how much SE is appropriate in specific circumstances to help ensure that the right amount of resources is available with the right skills. • Economics. Research how to better estimate systems engineering cost and schedule for complex programs and complex systems. • Assessment. Research improved process models and decision criteria for better SE support of management decisions across all life cycle phases (currently funded, in part, as the EM TTO)

16. 4. Program and Systems Engineering Integration - II • Maturity and Readiness. Research how to quantitatively assess the maturity of SE artifacts. Be able to inform technical, management, governance, and investment decisions using that assessment. • Teambuilding. Research how to better stimulate and support effective distributed team performance among SEs and other stakeholders. • System of Systems, Networks, and Enterprises. Research how to manage the acquisition and development of a SoS, a network-centric system, or an enterprise system where there is typically no single “owner” for all the component systems, components emerge and disappear dynamically, and components often have conflicting acquisition strategies. • Services. Research how to effectively manage the acquisition and development of a service as a function of type, scope and complexity, uncertainty, risk, and other characteristics.

17. 5. Life Cycle Systems Engineering Processes • Life cycle models. Research the best life cycle models for the acquisition and development of a SoS, a network-centric system, a service, or an enterprise system as in Thrust 4. • Balance. Research life cycle models for the acquisition and development of a system which begins as a quick response built using agile methods and then evolves into a system that is deployed on a larger-scale with radically different maintainability, availability, and similar attributes.

18. 5. Life Cycle Systems Engineering Processes - II • Architecting. Research how to create architectures that demonstrably and quantifiably support key system properties such as assurance, scalability, availability, producibility, interoperability, lethality, and resilience, and how to derive architectural component characteristics and constraints from desired system properties. • Landscape. Create a landscape of SE MPTs with respect to key characteristics such as their maturity, the types of systems to which they are best applied (e.g., network-centric vs. embedded or enterprise vs. platform), and their value in actual application. Use insights from this landscape to help guide where SE MPT research is needed. (currently funded, in part, as the MPT TTO)

19. SERC Research Initiation Strategy FY09 Focus SE external Factors/Context Mission Drivers Research to Address Specific SE issues Determine Modified MPTs To address gaps Validate MPT Effectiveness Research to Address Specific SE issues Determine Modified MPTs To address gaps Validate MPT Effectiveness Baseline SE MPTs Research to Address Specific SE issues Determine Modified MPTs To address gaps Validate MPT Effectiveness Research to Address Specific SE issues Determine Modified MPTs To address gaps Validate MPT Effectiveness Determine SE Effectiveness And Value Measures Early focus on a solid baseline and quantifiable, observable Measures to enable future demonstration of improvement

20. Outline • Nature of UARCs • SERC Overview • SERC organization and vision • SERC research strategy • Initial tasks • SysE Effectiveness Measures Assessment (EM) • SysE Methods, Processes, Tools Evaluation (MPT) • Workshop objectives and approach

21. First Two Task Orders “Assessing Systems Engineering Effectiveness in Major Defense Acquisition Programs (MDAPs)” Government lead: OUSD(AT&L)/SSE Barry Boehm (USC) task lead, with support from Stevens, Fraunhofer Center, University of Alabama at Huntsville “Evaluation of Systems Engineering Methods, Processes, and Tools (MPT) on Department of Defense and Intelligence Community Programs” Government lead: DoD Mike Pennotti and Rich Turner (Stevens) task leads, with support from USC, University of Alabama at Huntsville, USC, Fraunhofer

22. Coordinated approach • Coordinated management • Common collaborators, battle rhythm with regular e-meetings, shared workshops • Research Integrity Team • Coordinated Technical Approaches • Definitions, evaluation criteria and methods • Cross-feed/peer review ongoing results • Synergetic contribution to Sponsor’s SysE effectiveness goals • Best practices + progress monitoring and improvement • Common context • Domains of interest and levels of organization 22

23. Workshop Objectives and Approach • Review, improve on EM and MPT task objectives • Utility of results to DoD constituencies • Identification of needs-capabilities gaps • Identification of promising research directions • Review, improve on EM and MPT task approaches • Survey and evaluation approaches, criteria, and instruments • Involvement of key stakeholders • Contractors, program/portfolio managers, oversight organizations • Coverage of DoD application domains • Initial EM priority: Weapons platforms • Initial MPT priority: Net-centric services • Test and evaluation of results • Capture participants’ relevant experience

24. SERC Research Methodology

25. Example: Mission and StakeholderRequirements and Expectations Can we develop a transformative, interactive, and graphical environment to bring stakeholders (warfighters and analysts) together with SEs to develop a graphical/visual conops in an extremely agile manner? Every study on failed projects, refers to inadequate requirements, and understanding of the “real” problem: large projects and small projects; defense or commercial Understand and synthesize advances in multi-media technologies and interactive graphics; gaming technologies; real options theory