Strengthening Systems Engineering Execution at Boeing ….. And Meeting Customer Needs

1. Strengthening Systems EngineeringExecution at Boeing …..And Meeting Customer Needs Dev A. Banerjee Division Director – Systems and Flight Engineering (SAFE) Boeing St. Louis 6th Annual Systems Engineering Conference October 23, 2003

2. Outline • Assessment of USAF for better SE • Incentivizing Contractors (Excerpts from Dr. Sambur guidance) • Mapping of USAF SE Elements to PMBP, CMMI and SEPM • Joint DoD/Industry Suggested Actions • Strengthening SE Execution at Boeing • Establish Management Commitment • Provide Infrastructure • Monitor and Support Execution

3. Incentivizing Contractors for Better SEExcerpts from Dr. Sambur Memo • “An immediate transformation imperative for all our programs is to focus more attention on the application of Systems Engineering (SE) principles and practices throughout the system life cycle.” • “A more robust SE environment can only be achieved through joint cooperative efforts with our contractors.” “ SAF/AQ and all other MDAs will not sign out any future ASPs that lack the necessary and sufficient attention to SE.” • “Additionally, we are identifying ways to improve SE throughout the acquisition process, including workforce issues such as education and training,; tools such as policies, instructions, and guidance; and institutions such as the proposed Center of Excellence for SE under the Air Force Institute of Technology (AFIT).” • “---the Air Force leadership will take a strong and renewed interest in how our acquisition community addresses SE disciplines and practices.” Marvin Sambur Assistant Secretary of the Air Force (Acquisition) Policy Memo, dated April 09, 2003 USAF Committed to Robust SE Practice for Future Acquisitions

4. System(s) Requirements Development Mapping of Air Force SE Elements to PMBP, CMMI, and SEPM – existing pieces of our SE Infrastructure Sambur SE Elements (1) - Examples - Program Management CMMI Process Areas CMMI Process Area Best Practices to SEPM Section Systems Requirements Document Requirements Management Requirements Management 2.2 Requirements Development 1.2, 1.3, 1.4 Conduct and thoroughness of reviews Use of Independent Reviews Verification 1.2, 1.5, 1.6 Validation 1.5, 1.6 Test and Evaluation Master Plan (TEMP) Provides fully integrated engineering solutions Systems Integration (Proposed) Product Integration 1.5, 2.6 Effective SE interface control and management Interface Definition / Management (Proposed) Trades Studies (Proposed) Decision Analysis and Resolution 1.4 Affordability Technical Solution 1.2, 1.3, 1.4, 1.5, 1.6 System Safety Plan System Safety Risk Management Plan Risk Management Risk Management 1.4, 2.5 Configuration Management Plan Baseline Management Configuration Management 1.6, 2.6, 2.7 Systems Engineering Management Plan Organization Organizational Process Focus 3.1 Organizational Process Performance Organizational Innovation and Deployment 3.3 Integrated Project Management (IPPD) 2.1, 2.2, 2.3 Integrated Teaming 2.1 Life cycle management w/production and retrofit strategic plans Integrated Master Plan Integrated Planning Project Planning 2.1, 2.2, 2.6 Integrated Master Schedule Integrated Schedules Earned Value Management Project Monitoring and Control 2.2, 2.5 Source Selection Plan Supplier Management Supplier Agreement Management 2.4 Integrated Supplier Management System Maturity Matrix Technical Performance Measurement Measurement and Analysis 2.2 Closed Loop Corrective Action Causal Analysis and Resolution Contractor Performance Assessment Rating System Management Information System Quantitative Project Management 2.1 Help Needed All Program Management Best Practices Apply Organizational Process Definition 3.1 Organizational Training 3.2 Program Quality Plan Process and Product Quality Assurance 2.8 Logistics Management Plan Provide Support Organizational Environment for Integration 3.4 Customer informed of configuration changes and impacts Customer Communication/Contact Plan General Goals and Processes Program Management Best Practice Suggested new Program Management Best Practice Proposed new PMBP awaiting approval Quality and Mission Assurance Best Practice (1) Source Memo: "Incentivizing Contractors for Better Systems Engineering", dated 6 Jan 2003 from Dr. Marvin Sambur, Assistant Secretary of the Air Force for Acquisition

5. Suggested Actions from the Joint DoD / Industry Meeting on SE – 11 April 2003 • SE needs to be considered across the Program Life Cycle • Develop case studies on SE, with both good and bad program examples • Make use of disciplined assessment process to monitor rate and level of SE implementation • Recommend minimum or normal set of SE products or deliverables • Create a consistent set of SE measures/metrics • Define an appeal process and take corrective action if a program deviates from the minimum level of SE needed for sound engineering practice

6. Strengthening Systems Engineering Execution at Boeing I. Establish Management Commitment II. Provide Infrastructure • Command Media • VP Engineering guidance to Chief Engineers • EIB & PM Council direction to Bus • Goals/Metrics III. Monitor and Support Execution • Comprehensive SE process assets (processes, tools, templates, etc) • Training, SMEs • Management tools • SE HILT activity across sites • Oversight by SE HILT • - SE review of proposals • and programs • Disciplined assessment process • Metrics: • - Monitoring Rate • - Compliance Rate Corrective Action

7. I. Establish Management Commitment • Command Media • “Systems Engineering Process Requirements”- in-work • Systems Engineering Process Documents • Systems Engineering Process Manual • Leadership Endorsement / Direction • VP Engineering advisory to site Chief Engineers emphasizing commitment to SE • PM Council endorsement • Goal • Initiate effort to define goal/metric addressing SE execution Leadership Advisories and Goals will Reinforce Expectations for SE Implementation across Boeing

8. II. Systems Engineering Infrastructure • SE Process Assets, Management Tools • The SEPM outlines detailed requirements for effective SE process • CMMI to provide detailed set of process assets (process documentation, templates, etc) • Program Management Best Practices (PMBPs) address some key aspects of systems engineering • Gaps in Process Assets (e.g. handbooks in specialty areas) to be filled by SE Horizontal Integration Leadership Team (HILT) • Training, SMEs • Considerable training in SE process exists throughout Boeing • The SE HILT to consolidate SE Training • CMMI compliant process assets in development SE HILT Leadership, coupled with Site CMMI activities, will ensure a comprehensive SE infrastructure

9. Boeing Systems Engineering HILT / Process Action Teams (PATs) Huntington & Seal Beach Anaheim Canoga Park El Segundo Florida Houston Huntsville Long Beach Mesa / Philly Puget Sound St. Louis Wichita Systems Engineering HILT Process Action Teams (PATs) Network Systems Human Operations System of Survivability / Vulnerability Centric Info. Affordability R,M&T Safety Engineering Factors Analysis Systems & Comm. Strengthen/Integrate SE Across Boeing

10. Available SE Training for Network Centric-Info & Comm (Sample) Candidates • Info & Comm Arch • Networks • Info Operations • Adv Tech Courses • SE Processes • SoSE Process • DoD Sys Acq Process • Sys Arch Devel • UCLA Ext • UC Irvine • AFCEA • U of KS Sr. SEs; IPT Lds; PMs; CEs Project Teams; SE Specialists Everyone Source of Training Certificate Level Internal Process- Related Training Internal Domain/ Technical Training External Training Basic Inter Adv Training Basic Intermediate Advanced Formal Edu- Graduate - Workshops - Cross Trng SE LeaderExper (yrs) 4 2 6 4 1 7+ 4 2 # of Courses (Processes & Tools) Network Centric - Info & Comm SE Analysis & Control Affordability R, M&T Human System Interface System Safety Ops Analysis System of Systems (SoS) Survivability/ Vulnerability Advanced Intermediate Training Topic By SE Discipline Basic Training Level 4-8 8-13 13-18 SE Training Database Typical Guidance • SE Specialists and Technical Fellows more in-depth training in a selected domain. • IPT Leaders and Program Managers more general training across the SE disciplines.

11. Available Network Centric Training (Sample) Network - Centric Communications Training Program: Internal Process - Related Training SE Process Requirements Management Risk Management Trade Study Methodology System - of - Systems Systems Engineering Process Detailed System - of - Systems Process Systems OO/UML Process OO Modeling in UML OO Analysis and Design The DoD Systems Acquisition Process System Architecture Development High Level Architecture for Simulation - DII COE C4ISR Architecture Framework Principles and Applications UC Irvine – Certificate Program Armed Forces Communications & Electronics Association (AFCEA) We bsite courses: Network - Centric Communications Training Program: Topics Architectures Networks Information Operations Analog vs. Digital Comm Ø Ø Internet Ø Information assurance Ø Comm Spectrum Ø Interoperability Ø Information Warfare Ø SATCOM Ø Performance Ø C4ISR Ø Wireless Comm/ Ø Protocols/controls Ø Information Protection Terrestrial/ Cellular Ø Processing Techniques Ø Processing Techniques Ø Laser Com Ø Advanced/evolving technologies SoSE Tool Kit Ø Low Probability of Intercept Ø Vulnerabilities Ø Network protection Ø Information/data management Ø Evolving Technologies - jam Techniques Ø Anti Ø Performance In-house training plans focused in 2 areas: domain/technology training and process training Network Centric Communications Training Program External Training Network - Centric Communications Training Program: Universities / Associations / Gov. Labs / Agencies / Commercial Companies Internal Domain / Technology - Related Training UCLA Extension – Short Course Programs Communications; Network Technology; Space Mission Analysis Satellite Communications Payloads & Systems Design Information & Communication Architectures High Frequency Satellite Communications Satellite Communications Systems Open Architectures Phased Array Antennas Communications & Network Engineering (18 units required) Laser Communications Advanced Communications Systems Engineering (18 units required) Software Architecture RF Subsystem Design http://www.afcea.org/education/index.htm - – Networks individual short courses (3 6 days) Aviation & Electronics Schools of America - Network Centric Warfare Fiber Optics Web Services Systems for RF Telecommunications Network Technologies and Practices Information Operations Defense Information Systems Agency (DISA) offers COE course Government Information Technology Thrust University of Kansas - the Information and Telecommunication Technology Center: Modulation Techniques Lawrence Livermore National Laboratory Working with C4ISR Software/Commercial Companies offering related technology/softwa re training: Fundamentals of Digital Signal Processing , etc. Robelle, Sun Microsystems, Borland USA, Object Management Group Time and Frequency Reference Concepts Industry Seminars Advanced Technology Courses - Cisco Networking Advanced Satellite Communications Payloads - Alta Vista Search/MetaData - Satellite Network Architectures Topia - Intelligent Agents Advanced Fiber Optics Entrust - Security .

12. III. Monitor and Support Execution on Proposals / Programs • Sites provide resources to execute SE and monitor execution. • SE HILT provide ‘integration mechanism across Boeing’ • Two Key Metrics to be monitored: • - “Monitoring Rate” Metric - “Implementation Level” Metric • SE Evaluation Templates required for program reviews: • - proposal coverage - kickoff planning - program in-process reviews • 3. SE HILT analyze metrics with sites; Jointly conduct root cause analyses and develop Corrective Action Plans • 4. SE HILT monitor implementation & results of corrective actions • Effective monitoring • Root cause analysis • Implementation of corrective • action will strengthen the execution of Systems Engineering… consistent with customer desires.

13. System Life Cycle RFP Pre-Concept / Requirements Development Concept Definition Preliminary Definition Detail Definition Support Production Proposal Start Program Start SRR PDR CDR Conduct In-Progress Reviews on Programs Ensure SE Integrity Through Tailored Reviews during Major Program Milestones

14. The Metrics: - “Monitoring Rate” Metric: Was the SE Process Monitoring Activity conducted? - Measure instances as % of opportunities. Example: At each site, measure the number of proposals reviewed for SE compliance at Gate C as a percentage of the total number of Gate C proposals reviewed by the site. - “Implementation Level” Metric: When the SE Process Monitoring Activity was conducted, what was the level of execution implementation? Example: For each proposal reviewed, evaluate the level of coverage of SE process execution.

15. Boeing is Committed to SEExecution across the Enterprise