Systems Engineering – A Global Perspective Systems Engineering Seminar Series

1. Systems Engineering – A Global PerspectiveSystems Engineering Seminar Series Dinesh Verma, Ph.D. Professor and Associate Dean Stevens Institute of Technology dverma@stevens.edu

2. Underlying “TONE” for this Presentation (Actually, Cynical) Idealistic or Realistic The Contextual Setting for this Presentation “Rear View Looking” or “Forward Looking”

3. Systems Engineering – Expectation • Successful implementation of proven, disciplined systems engineering processes results in a total system solution that is: • Robust to changing technical, production, and operating conditions; • Adaptive to the needs of the users; and • Balanced among the multiple requirements, design considerations, design constraints, and program budgets. 3

4. Some Inhibitors to Good Systems Engineering: Based on a survey of IT architects and project managers • Customer Related Input: • Isolation from real “user” • Customer requirements and (even) identity not clear • Customer doesn’t know what they want • Scope creep; Undocumented system scope and functionality • User/buyer too distant • Don’t understand the customer value system • Management Related Input: • Executive management doesn’t buy in • Lack of teamwork • Program Managers not empowered • Program manager and capture managers are different • Unstable funding stream; Lack of upper management support • Organizational/Cultural Input (Some Perceptions): • SEA only adds to the Project Cost • SEA often seen as an “outside” team or “project reviewer” role We would like you to build us a lawn mower please! 4

5. Deploying Systems Engineering within a Commercial Global Leader: Some Results

6. 1st project uses SE principles SE organization introduced SE Reviews / Scorecards introduced Directive to use SE on projects >$1M ‘Fundamentals of SE’ course introduced 1st commercial account uses SE Directive to use SE on projects > $500K Formal SE dept created Systems Engineering Has Been Applied to Both Internal and Commercial Accounts 2000 2001 3rd qtr 2nd qtr 1st qtr 1st qtr 3rd qtr 4th qtr 2nd qtr 4th qtr 2002 2003 3rd qtr 2nd qtr 1st qtr 1st qtr 3rd qtr 4th qtr 2nd qtr 4th qtr SE process integration - GS Method 13 projects using SE SE process integration - AMS MS SEI introduces CMMI 1.1 ‘SE Design’ Class introduced SE team grows to 30 12 completed and 13 active projects using SE SE deliverables templates provided 17 completed and over 50 active projects using SE Over 230 trained in SE Fundamentals SE team grows to 14 SE process updated for CMMI

7. Systems Engineering Process defines deliverables and a series of Reviews (Part I) Need / Opportunity Identification Conceptual System Specification Component Architecture Detailed Design Customer Baseline System Baseline Architecture/Component Baseline Design Baseline Systems Requirements Review (SRR) Business Requirements Review (BRR) Preliminary Design Review (PDR) Critical Design Review CDR) Systems Req’ment Specs Component Req’ment Specs Component RTVM Component Level Architecture Component Design System Level Architect. Component Test Plan Business Require. Specs. Test Architecture RTVM Customer Provided Systems Engineering Provided Component Developer Provided

8. Comp. Design Comp. Test Plan Systems Engineering Process defines deliverables and a series of Reviews (Part II) Development New Production System Test and Production System Update Detail Design Design Baseline Test Baseline Production Baseline Test Readiness Review (TRR) Production Readiness Review (PRR) CDR System Test Plan / Test Cases System Test Strategy Deployment Plan Release Content System Test Data Data Migration Plan Test Traceability Matrix. Move to Prod. Plan Customer Provided Component Developer Provided Systems Engineering Provided Service Delivery / Managed Ops Provided System Test Provided

9. ISM delivered 5% under budget and with higher quality in production The charts here are based on data collected from a recent study analyzing project defects by type and phase. Here ISM defects by phase is compared to 46 similarly sized projects not utilizing SE. Total defect counts for non-SE projects exhibited 53.4% of total project defects during the Test Phase of the project. On ISM defects were detected earlier in the project life-cycle. In fact 56% of ISM detects were detected in Plan Phase. The chart on the left illustrates the cost implications of early defect detection as found with ISM 2.0. In effect ISM 2.0 expended 2.4 timeslessthan what would have normally been required for the non-SE oriented projects compared to in the study.

10. IGA Metrics show 8% cost avoidance when comparing SE&A projects to non-SE&A projects = 15% of $10.7MM Baseline = 7% of $10.7MM Baseline = 8% Cost Avoidance

11. Similar Initiatives Underway at… 11

12. R&D spend per product launched is much higher than key competitors • Cost per product program is much higher than major competitors • high R&D spend as % of sales • absolute spend >2* any other • not many more products launched than other players with much less R&D spend • (benchmarking takes into account broader scope and uses comparable definition of what is a product) • Higher sales volumes per product keep R&D costs per unit shipped in line • no clear advantage in terms of product competitiveness • higher volumes come from other resources: global reach; brand’ strong channels; logistics • No economies of scale from R&D despite spend more than twice any other player

13. Despite much higher spend, products now lag best competitors on key dimensions Nokia versusAverage of Competitors Nokia versusBest Competitors - 3 - 2 - 1 0 1 2 3 - 3 - 2 - 1 0 1 2 3 D e s i g n / s t y l e / a p p e a r a n c e - 2 D e s i g n / s t y l e / a p p e a r a n c e 0 Competitors now have better designs P r i c e - 1 P r i c e 0 S i z e & w e i g h t - 1 S i z e & w e i g h t 0 B a t t e r y p e r f o r m a n c e 0 B a t t e r y p e r f o r m a n c e 1 B u t t o n s a n d k e y p a d 0 B u t t o n s a n d k e y p a d 0 0 D i s p l a y D i s p l a y 0 D u r a b i l i t y / b u i l d q u a l i t y 0 D u r a b i l i t y / b u i l d q u a l i t y 1 0 1 F e a t u r e s / f u n c t i o n s F e a t u r e s / f u n c t i o n s 0 2 Q u a l i t y o f r e c e p t i o n Q u a l i t y o f r e c e p t i o n Nokia still leads on ease of `phone user 0 2 R e s i s t a n c e t o s c r a t c h e s R e s i s t a n c e t o s c r a t c h e s 0 1 T e c h n i c a l r e l i a b i l i t y T e c h n i c a l r e l i a b i l i t y 1 2 E a s e o f m e s s a g i n g E a s e o f m e s s a g i n g 1 2 E a s e o f n a m e s e a r c h E a s e o f n a m e s e a r c h 1 2 M e n u s y s t e m M e n u s y s t e m Zero = about same as competitors, 1 = 1 standard deviation better than competitors, and so on Competitors: Motorola, Panasonic, Sagem, Samsung, Sharp, Siemens and SonyEricsson

14. Key Concerns Identified – Symptoms R&D Productivity Impacted Innovation Impacted Morale Impacted • Asset management and technology management is a weakness • Architecture portfolio management is extremely poor • Design consistency is missing • The norm seems to be reactive design versus proactive design • Architecture studies, tradeoffs, and evolution could be better planned (architecture archeology!) • Architecture ownership is not clear, and often not executed • Too many product/platform variants, each requiring special testing and support • Reactive redesign during implementation – We don’t know the performance thresholds of our architectures • Architecture is often the result of implementation rather than the driver • Too much discretion at the lowest implementation levels • Example: Data and image formats • Software quality could be better – expensive rework currently required • Requirements are not managed (end to end) • Competing and conflicting positions in external forums • Requirements management (traceability) is often poor • Interface management is poor • Not enough linkage to business requirements Sometimes I feel like I am in a different company than them… Business priorities don’t seem to be linked to Technology priorities” I wish I had tools to help my management team realize the importance of architecting… short-term-ism gets in the way… our headaches today were born 1.5 to 2 years ago… If these are symptoms, then what are the causes?

15. Resulting Situation • Systems and devices often developed and tested through significant individual efforts and human networks – under significant schedule pressure • Existing processes are often uniquely (one time use) tailored/used • Reuse is a myth in some cases, and getting in the way of innovation in other cases (because of serious schedule pressures) • Processes and methodologies (and even language) are personality dependent, with no time for documentation (competency development?) • In the absence of clear organizational authority and accountability, personality based authority and accountability has evolved in the current situation (e.g., Roles and responsibilities between BGs and TPs) • CHALLENGE: Scale up NOKIA R&D efficiency (Time to Market) and innovation (Feature Leadership), without scaling up in size and compromising quality • Architectural thinking can contribute

16. Theory versus (Virtual) Reality…Primary Reasons for Dysfunctional Behavior – My Opinion • Confusion between “What you NEED” versus “What you WANT” • Also called Gold-Plating • It is the moral duty of a systems engineer to articulate the resulting cost and schedule delta • Confusion with regard to the SYSTEM BOUNDARY • This is more difficult for legacy systems with undocumented and implied interfaces; and even more so for “network-centric systems” and “SoS” • Confusion (?) with regard to fidelity between the technical project scope and its allocated budget and schedule • The result is cynicism and complacency, along with other negative behavioral patterns • Lack of Leadership

17. Holistic Thinking versus Local Thinking… Innovation Associates 0943-98 17

18. A few words about competency development…

19. Discipline Centric Systems Engineering Programs: These are programs where the major is designated only as Systems Engineering Domain Centric Systems Engineering Programs: These are programs where the major is designated as X and Systems Engineering; or Systems and X Engineering. In this case, the most common instances of “X” Engineering are: Industrial Engineering Manufacturing Engineering Electrical Engineering Management Engineering Computer Engineering Academic Perspective: Discipline and Domain Centric Systems Engineering Programs The primary source of this data is: Fabrycky, W.J., “Systems Engineering Education in the United States”, Proceedings, Conference on Systems Integration (CSI), Stevens Institute of Technology, New Jersey, March 2003.

20. Academic Perspective: Universities with Discipline Centric Systems Engineering Programs - 30 Air Force Institute of Technology California State UniversityColorado School of MinesCornell UniversityGeorge Mason UniversityGeorge Washington UniversityIowa State UniversityJohns Hopkins UniversityNational Technological University Naval Postgraduate SchoolOakland UniversityPolytechnic University - FarmingdalePortland State UniversityPurdue UniversityRochester Institute of Technology Southern Methodist University Stevens Institute of Technology University of Alabama - Huntsville University of ArizonaUniversity of Idaho University of Illinois at Urbana-ChampaignUniversity of Maryland University of MassachusettsUniversity of MinnesotaUniversity of Missouri - RollaUniversity of PennsylvaniaUniversity of Rhode IslandUniversity of Southern California University of VirginiaVPI and State University The list in the primary reference contains 35 records. Four of these referred to Universities with only an Undergraduate Program in Systems Engineering, and one to a University with only a Doctoral Program in Systems Engineering.

21. Academic Perspective: Universities with Domain Centric Systems Engineering Programs - 38 Auburn University Boston University California State University - Fullerton Case Western Reserve University Georgia Tech Massachusetts Institute of Technology New Jersey Institute of Technology North Carolina A and T University Northeastern University Ohio State University Ohio University Polytechnic University Purdue University Rensselear Polytechnic University Rutgers, The State University San Jose State University Stanford University Texas Tech University University of Alabama - Huntsville University of Arizona University of Central Florida University of Connecticut University of Florida University of Houston University of Illinois University of Memphis University of Michigan Ann Arbor University of Michigan-Dearborn University of Minnesota University of Pittsburgh University of Rhode Island University of South Florida University of Southern California University of Southern Colorado University of St. Thomas Virginia Tech Wichita State University Youngstown State University

22. 30 + 38 ~15 Industry/Government Perspective: Systems Engineering Education and Training • Definition of Relevant: • Relevance of the curriculum – orientation to DoD projects and programs • Portability and flexibility of the delivery format – distributed organizations • Hybrid – credit and continuing education • Basis: • The Boeing SE Education Program (50 > 15 > 6 > 2) • A DoD Component SE Education Program (80 > 25 > 11 > 2) Available… Relevant…

23. 30 + 38 ? ~15 Industry/Government Perspective: Systems Engineering Education and Training • Definition of Critical Mass: • Number of Tenured or Tenure Track Faculty • Number of Faculty with DoD/Aerospace Relevant Project/Program Experience • Bench Strength …Why? Available… Relevant… Critical Mass…

24. 30 + 38 ? ~15 Industry/Government Perspective: Systems Engineering Education and Training • Definition of Critical Mass: • Number of Tenured or Tenure Track Faculty • Number of Faculty with DoD/Aerospace Relevant Project/Program Experience • Bench Strength Available… Relevant… Critical Mass… It is my opinion that we do not have a critical mass in graduate SE education in the US… However, we do have a very mature base and recognition within industry and government to facilitate the building of this critical mass…

25. Systems Engineering Education: Relevant graduate level courses, anchored with academic rigor and yet sensitive to current system development and integration challenges. Flexible delivery formats – online/distance; modular Flexible for part-time course loads Full time leave of absence for educational purposes is a thing of the past… Systems Engineering Training: Relevant and focused training courses on specific subjects of immediate relevance Industry/Government Expectations: Systems Engineering Education, Training, Education • Systems Engineering Research: • Development of an SE Toolkit (Templates, Metrics, etc.) of immediate utility within an organization • Domain centric • Specific purpose of enhancing development efficiency and effectiveness • Usage centric • Architectural assessment frameworks • Open systems (a much abused phrase) • Domain centric • Others… In an environment of high demand and scarce supply, there are many experts… but who sets the minimum thresholds on relevance and quality?

26. First Quarter Second Quarter Third Quarter Fourth Quarter Decide program projects SDOE-651 SDOE-780 SDOE-606 SDOE-655 Equivalent of 20% facilitation support throughout 2004 Crossing the boundaries… the “Open Academic Model” • Blurring the boundary between academia and industry/government • Bringing a “fresh” perspective to industry/government in an executable form – a specific method, tool, heuristic, template • Bringing industry/government “reality” into academia in a researchable or usable form – a problem statement, a specific challenge, guest instructors, heuristics, case studies The Challenge from Ralph Nelson at IBM Global Services

27. Wrap-up: Essential Elements of a Systems Engineering Program • Leadership • Policy with Executive Measurements • Investment to develop the process, templates, education, mentoring • Process and tools • Defined Process • Templates • Skilled SEs – Core group of SEs with 15 years experience on major programs within relevant domains • Certification Program • Education • Experience • Examination • Ongoing Process Improvement