Systems Engineering – A Perspective

1. Systems Engineering – A Perspective Dinesh Verma, Ph.D. Professor and Associate Dean Stevens Institute of Technology dverma@stevens.edu

2. Simple Definition… • Systems Engineering is “problem solving and solution delivery.” A key prerequisite to good “problem solving” is good “problem definition.” Now this has other prerequisites! • Some best practices: • Translating customer needs (business and technical) into acceptance criteria - 5 to 7 critical customer requirements agreed to in measurable/testable form. • Identifying requirements and then managing them (and tracing them) through the subsequent development, integration, testing phases. • Translating the requirements into an “architecture” that becomes a “linkage” between what the customers want and what the developers will build and test. • Developing a test architecture, test plans and procedures that are traceable to the requirements for maximum focus and efficiency Sounds very simple! A lot of organizations have developed processes that attempt to capture the above intent. But very few are able to execute it…

3. System Design Life Cycle Models:An Automotive Example (VOLVO Car Corporation) 3

4. System Design Life Cycle Models:A Telecom Example (NOKIA Networks) Capability for Volume Deliveries Program initiated Program proposal ready Program plan ready Ready for integration Ready for verification Ready for Ramp-up E-1 E0 E1 E2 E3 E4 E5 Design and implement Implement and integrate Verify Ramp-up Define Plan and specify Program allowed to start using resources Program established Commitment of features, resources and milestone dates. Specification done Volume deliveries can start Real HW done and HW in maintenance mode. HW and SW main verification starts. SW is module tested and proof on product functionality exist (=SW implementation ready). "Proof of concept" * HW implemented. Real HW and basic/ low level SW integrated and core functionality works. Idea of performance exists. First SW build made. Proof of product architecture. Traditional Pilot deliveries start. HW and SW have been tested together and released as a product Program main contents frozen for program planning purposes (optional) Requirements specs done Product Design frozen (optional) Program completed (optional) Trial deliveries can start (Optional) Functional tests done and HW fullfills legal type approval requirements E0.5 E1.5 E 3.5 E 5.5 Optional Milestones can be moved. I.e. E1 and E1.5 dates can be the same. * Core functionality can be I.e. control plane, signal goes through (typically not call yet). Exact contents of core functionality is need to be defined in E1 4

5. System Design Life Cycle Models:A Workstation Example (SUN Microsystems) 5

6. The IBM AMS Systems Engineering Process defines deliverables and a series of Reviews (I) Need / Opportunity Identification Conceptual System Specification Component Architecture Detailed Design Customer Baseline System Baseline Architecture/Component Baseline Design Baseline Systems Requirements Review (SRR) Business Requirements Review (BRR) Preliminary Design Review (PDR) Critical Design Review CDR) Systems Req’ment Specs Component Req’ment Specs Component RTVM Component Level Architecture Component Design System Level Architect. Component Test Plan Business Require. Specs. Test Architecture RTVM Customer Provided Systems Engineering Provided Component Developer Provided 6

7. Comp. Design Comp. Test Plan The IBM AMS Systems Engineering Process defines deliverables and a series of Reviews (II) Development New Production System Test and Production System Update Detail Design Design Baseline Test Baseline Production Baseline Test Readiness Review (TRR) Production Readiness Review (PRR) CDR System Test Plan / Test Cases System Test Strategy Deployment Plan Release Content System Test Data Data Migration Plan Test Traceability Matrix. Move to Prod. Plan Customer Provided Component Developer Provided Systems Engineering Provided Service Delivery / Managed Ops Provided System Test Provided 7

8. SE in the System Life Cycle“The Wall Chart” 8

9. Key Concepts… • The notion of Systems Thinking and Contextual Setting • The notion of Baselines • Facilitates requirements definition and management, change management, and scope control • The notion of Functional (Dynamic) And Non-Functional (Static) Requirements • A holistic view that addresses new or modified Business Processes, while also addressing concerns pertaining to Performance, Availability and Reliability, Scalability and Security • The notion of Baseline Reviews and Objective Metrics • An in-process review to validate completeness of the baseline and identify risks and defects • The notion of the System Life Cycle (Define – Design – Build – Test – Deploy – Operate – Upgrade/Retire) • Systems Engineering should have “ownership” and accountability through the various phases of the life cycle.

10. “I try to be very clear in separating the “problem space” from the “solution space”.” Chief Architect, NOKIA Technology Platforms“I try to design at three levels: my level, the level above me, and the level below me.” Chief Architect, L3 Communications“I architect and design in two iterations. In the first, my focus is on feasibility and completeness, while in the second, my focus is on “goodness” and other refinements and embellishments such as scalability, modularity, and such.” Chief Architect, IBM Global Services 10

11. Systems Engineering – Expectation • Successful implementation of proven, disciplined systems engineering processes results in a total system solution that is: • Robust to changing technical, production, and operating conditions; • Adaptive to the needs of the users; and • Balanced among the multiple requirements, design considerations, design constraints, and program budgets. 11

12. Systems Engineering – Expectation • Successful implementation of proven, disciplined systems engineering processes results in a total system solution that is: • Robust to changing technical, production, and operating conditions; • Adaptive to the needs of the users; and • Balanced among the multiple requirements, design considerations, design constraints, and program budgets. 12

13. DoD Systems Engineering Shortfalls* • Root cause of failures on acquisition programs include: • Inadequate understanding of requirements • Lack of systems engineering discipline, authority, and resources • Lack of technical planning and oversight • Stovepipe developments with late integration • Lack of subject matter expertise at the integration level • Availability of systems integration facilities • Incomplete, obsolete, or inflexible architectures • Low visibility of software risk • Technology maturity overestimated Major contributors to poor program performance * DoD-directed Studies/Reviews 13

14. Some Inhibitors to Good Systems Engineering: Based on a survey of IT architects and project managers • Customer Related Input: • Isolation from real “user” • Customer requirements and (even) identity not clear • Customer doesn’t know what they want • Scope creep; Undocumented system scope and functionality • User/buyer too distant • Don’t understand the customer value system • Management Related Input: • Executive management doesn’t buy in • Lack of teamwork • Program Managers not empowered • Program manager and capture managers are different • Unstable funding stream; Lack of upper management support • Organizational/Cultural Input (Some Perceptions): • SEA only adds to the Project Cost • SEA often seen as an “outside” team or “project reviewer” role We would like you to build us a lawn mower please! 14

15. Theory versus (Virtual) Reality…Primary Reasons for Dysfunctional Behavior – My Opinion • Confusion between “What you NEED” versus “What you WANT” • Also called Gold-Plating • It is the moral duty of a systems engineer to articulate the resulting cost and schedule delta • Confusion with regard to the SYSTEM BOUNDARY • This is more difficult for legacy systems with undocumented and implied interfaces; and even more so for “network-centric systems” and “SoS” • Confusion (?) with regard to fidelity between the technical project scope and its allocated budget and schedule • The result is cynicism and complacency, along with other negative behavioral patterns • Lack of Leadership

16. Leadership at the Individual Level… • Leadership and Focus • Honesty • Openness – Lexicon (Key Issue) • Domain Knowledge – This is the key ingredient missing in most systems engineers today, resulting in significant impact to the credibility of the discipline • Ability for Systems Thinking • A very strong mental model, in the face of: • A diverse lexicon • Need for rough estimates, and to identify outliers • Changing standards (are they really changing?) • New fads (6-sigma, IPPD, Concurrent Engineering, Simultaneous Engineering) • Changing life cycle models (incremental, spiral, V, etc.) 16

17. Deploying Systems Engineering within a Commercial Global Leader: Some Results

18. 1st project uses SE principles SE organization introduced SE Reviews / Scorecards introduced Directive to use SE on projects >$1M ‘Fundamentals of SE’ course introduced 1st commercial account uses SE Directive to use SE on projects > $500K Formal SE dept created Systems Engineering Has Been Applied to Both Internal and Commercial Accounts 2000 2001 3rd qtr 2nd qtr 1st qtr 1st qtr 3rd qtr 4th qtr 2nd qtr 4th qtr 2002 2003 3rd qtr 2nd qtr 1st qtr 1st qtr 3rd qtr 4th qtr 2nd qtr 4th qtr SE process integration - GS Method 13 projects using SE SE process integration - AMS MS SEI introduces CMMI 1.1 ‘SE Design’ Class introduced SE team grows to 30 12 completed and 13 active projects using SE SE deliverables templates provided 17 completed and over 50 active projects using SE Over 230 trained in SE Fundamentals SE team grows to 14 SE process updated for CMMI

19. Systems Engineering Process defines deliverables and a series of Reviews (Part I) Need / Opportunity Identification Conceptual System Specification Component Architecture Detailed Design Customer Baseline System Baseline Architecture/Component Baseline Design Baseline Systems Requirements Review (SRR) Business Requirements Review (BRR) Preliminary Design Review (PDR) Critical Design Review CDR) Systems Req’ment Specs Component Req’ment Specs Component RTVM Component Level Architecture Component Design System Level Architect. Component Test Plan Business Require. Specs. Test Architecture RTVM Customer Provided Systems Engineering Provided Component Developer Provided

20. Comp. Design Comp. Test Plan Systems Engineering Process defines deliverables and a series of Reviews (Part II) Development New Production System Test and Production System Update Detail Design Design Baseline Test Baseline Production Baseline Test Readiness Review (TRR) Production Readiness Review (PRR) CDR System Test Plan / Test Cases System Test Strategy Deployment Plan Release Content System Test Data Data Migration Plan Test Traceability Matrix. Move to Prod. Plan Customer Provided Component Developer Provided Systems Engineering Provided Service Delivery / Managed Ops Provided System Test Provided

21. Successful implementation of System Engineering needs… • A “productized” process for efficient implementation • Globally consistent templates and processes, • Uniform and consistent metrics and lexicon (part of the SE culture) • Focus must be on the “necessary” and critical subset of the overall methodology and theory (Flexibility and Adaptability) • Tailoring for time-to-market considerations • Tailoring for schedule and resource considerations • Risk tolerance must be explicitly considered in the tailoring process • Implementation must be organizationally supported and nurtured • Linkage to strategic organizational goals is key • A well managed competency development program and a “community of practice”

22. ISM delivered 5% under budget and with higher quality in production The charts here are based on data collected from a recent study analyzing project defects by type and phase. Here ISM defects by phase is compared to 46 similarly sized projects not utilizing SE. Total defect counts for non-SE projects exhibited 53.4% of total project defects during the Test Phase of the project. On ISM defects were detected earlier in the project life-cycle. In fact 56% of ISM detects were detected in Plan Phase. The chart on the left illustrates the cost implications of early defect detection as found with ISM 2.0. In effect ISM 2.0 expended 2.4 timeslessthan what would have normally been required for the non-SE oriented projects compared to in the study.

23. IGA Metrics show 8% cost avoidance when comparing SE&A projects to non-SE&A projects = 15% of $10.7MM Baseline = 7% of $10.7MM Baseline = 8% Cost Avoidance

24. Discipline Centric Systems Engineering Programs: These are programs where the major is designated only as Systems Engineering Domain Centric Systems Engineering Programs: These are programs where the major is designated as X and Systems Engineering; or Systems and X Engineering. In this case, the most common instances of “X” Engineering are: Industrial Engineering Manufacturing Engineering Electrical Engineering Management Engineering Computer Engineering Academic Perspective: Discipline and Domain Centric Systems Engineering Programs The primary source of this data is: Fabrycky, W.J., “Systems Engineering Education in the United States”, Proceedings, Conference on Systems Integration (CSI), Stevens Institute of Technology, New Jersey, March 2003.

25. Academic Perspective: Universities with Discipline Centric Systems Engineering Programs - 30 Air Force Institute of Technology California State UniversityColorado School of MinesCornell UniversityGeorge Mason UniversityGeorge Washington UniversityIowa State UniversityJohns Hopkins UniversityNational Technological University Naval Postgraduate SchoolOakland UniversityPolytechnic University - FarmingdalePortland State UniversityPurdue UniversityRochester Institute of Technology Southern Methodist University Stevens Institute of Technology University of Alabama - Huntsville University of ArizonaUniversity of Idaho University of Illinois at Urbana-ChampaignUniversity of Maryland University of MassachusettsUniversity of MinnesotaUniversity of Missouri - RollaUniversity of PennsylvaniaUniversity of Rhode IslandUniversity of Southern California University of VirginiaVPI and State University The list in the primary reference contains 35 records. Four of these referred to Universities with only an Undergraduate Program in Systems Engineering, and one to a University with only a Doctoral Program in Systems Engineering.

26. Academic Perspective: Universities with Domain Centric Systems Engineering Programs - 38 Auburn University Boston University California State University - Fullerton Case Western Reserve University Georgia Tech Massachusetts Institute of Technology New Jersey Institute of Technology North Carolina A and T University Northeastern University Ohio State University Ohio University Polytechnic University Purdue University Rensselear Polytechnic University Rutgers, The State University San Jose State University Stanford University Texas Tech University University of Alabama - Huntsville University of Arizona University of Central Florida University of Connecticut University of Florida University of Houston University of Illinois University of Memphis University of Michigan Ann Arbor University of Michigan-Dearborn University of Minnesota University of Pittsburgh University of Rhode Island University of South Florida University of Southern California University of Southern Colorado University of St. Thomas Virginia Tech Wichita State University Youngstown State University

27. 30 + 38 ~15 Industry/Government Perspective: Systems Engineering Education and Training • Definition of Relevant: • Relevance of the curriculum – orientation to DoD projects and programs • Portability and flexibility of the delivery format – distributed organizations • Hybrid – credit and continuing education • Basis: • The Boeing SE Education Program (50 > 15 > 6 > 2) • A DoD Component SE Education Program (80 > 25 > 11 > 2) Available… Relevant…

28. 30 + 38 ? ~15 Industry/Government Perspective: Systems Engineering Education and Training • Definition of Critical Mass: • Number of Tenured or Tenure Track Faculty • Number of Faculty with DoD/Aerospace Relevant Project/Program Experience • Bench Strength …Why? Available… Relevant… Critical Mass…

29. 30 + 38 ? ~15 Industry/Government Perspective: Systems Engineering Education and Training • Definition of Critical Mass: • Number of Tenured or Tenure Track Faculty • Number of Faculty with DoD/Aerospace Relevant Project/Program Experience • Bench Strength Available… Relevant… Critical Mass… It is my opinion that we do not have a critical mass in graduate SE education in the US… However, we do have a very mature base and recognition within industry and government to facilitate the building of this critical mass…

30. A Fundamental Dilemma • We’ve forgotten (and are continuing to forget) much of the systems engineering we once knew. • Reversing this trend has become an imperative for both Government and commercial industry. and yet… • What we once knew may be inadequate for solving the systems engineering problems we now face. 30

31. What do I mean… • Security in the Port of NY & NJ • Ten threat scenarios were defined • Each is low-likelihood, high-consequence • What is the eleventh? The Paradox of Ownership and Boundaries… The Driver and the Driven – The System and the Organization… Top Down…

32. Thanks! Dinesh Verma dverma@stevens.edu