MDA QSMA Solutions

1. MDA QSMA Solutions Mr. Edward Jopson Assistant Director MDA/QSI CQSDI March 27 2007

2. MDA PROGRAM CHALLENGES KI C2BMC GM AB Developmental Systems morphs to Operational System – Reliability standards are inherently more stringent Heritage Programs – Different mindsets, focuses and cultures, integration of these Heritage Programs into a BMDS AL STSS TH

3. MDA PROGRAM CHALLENGES (CONT’D) Acquisition Reform - Performance-based specifications and “Best Practices” replacing MIL-STDs Spiral Development - Qualification baseline changes and lack of test data to validate models

4. MDA SAFETY, QUALITY AND MISSION ASSURANCE DIRECTORATE MDA/QS DESCRIPTION • Started October 2002 • Currently encompasses 134 personnel in 28 locations • Supplier facilities, Test sites, Government facilities • Direct Report to MDA Director ROLES AND RESPONSIBILITIES • Develop and maintain within MDA a world-class Safety, Quality and Mission Assurance (SQMA) organization • Provide BMDS Elements with robust and practical policy, guidance and expertise • Develop an integrated BMDS SQMA team across MDA, the program elements, and the Ballistic Missile Defense industry • Provide MDA personnel and resources to assist Elements and Industry in SQMA activities • Terminate any operation that presents an immediate and unacceptable risk to personnel, property, or mission operations

5. MDA/QS Director Quality, Safety And Mission Assurance (QS) Assistant Director Quality (QSQ) Assistant Director Safety (QSS) Assistant Director Mission Assurance (QSA) Assistant Director Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

6. QS FIELD LOCATIONS MARs, BSOs & SAFETY PERSONNEL ATK, Magna, UT Lockheed, Denver, CO Ft. Greely, AK Ft. Greely, AK ABL, Kirtland AFB, NM JNIC, CO BSOs (6) Eagle Picher, Joplin, MO Raytheon, Woburn, MARaytheon, Andover, MA Kodiak Facility, AK Aerojet, Sacramento, CA Lockheed, Moorestown, NJ Lockheed, Sunnyvale, CA ATK, Elkton, MD Vandenberg AFB, Lompoc, CA MDA HQ Northrop Grumman, Redondo Beach, CABoeing, Anaheim, CA Edwards AFB, CA KI, Huntsville, ALGMD, Huntsville, ALTHAAD, Huntsville, AL GMD, Huntsville, ALGMD/IM & CTF, Huntsville, AL Orbital, Chandler, AZSpectrum Astro, Gilbert, AZ Raytheon, Camden, AR Raytheon, Tucson, AZ Honeywell, Clearwater, FL Black = QSLBlue = MARGreen = BSORed = Safety

7. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

8. MDA ASSURANCE PROVISIONS (MAP) • A standardized set of 14 SQMA provisions, requirements, and processes for mission and safety critical items: • Management • Design and Development • Software and Firmware • Technical and Mission Assurance Reviews • Reliability, Maintainability, and Availability • Parts and Materials Control Program • Integrated Test and Evaluation Program • Test, Measuring and Diagnostic Equipment and Standards • Interface Management • Configuration Management • Control of Nonconforming Items and Materials • Fabrication and Quality • Supplier Management • Safety

9. MDA ASSURANCE PROVISIONS (MAP) (CONT’D) • Provide methods to measure, verify, and validate mission success through: • Collection and analysis of metrics • Risk Assessment, technical evaluations, independent assessment, reviews • Conducting tests • Promote continuous process improvement to reduce cost, improve productivity, and assure SQMA • Specific applicability is defined in Mission Assurance Implementation Plan (MAIP) – Verification Matrix MAP is on/almost on all MDA contracts

10. MDA ASSURANCE PROVISIONS (MAP) COOPERATIVE APPROACH IMPLEMENTATION Near-Term Mid-Term Long-Term MDA Assurance Provisions Evaluate / Understand Element MA Practices Improve Element MA Programs Build Integrated MA Team Throughout MDA Define / Implement Mission Assurance Program For The BMDS Refine And Improve MAP And MDA MA Processes Mandate Use Of “Golden Round” Philosophy – Test Like You Operate Flow Down MA Requirements To Contractors And Vendors Formally Incentivize Mission Success MAP As A Core Standard On All New Contracts Perform Independent Reviews • Matrix QS Personnel Into Elements • Perform Gap Analysis • Collect / Share / Apply Lessons Learned • Prioritize High Risk Items • Independent Assessment • QS Personnel Matrixed Throughout MDA Elements • Provide Comprehensive “Second Look”

11. MISSION ASSURANCE IMPLEMENTATION IDO Design And Development Manufacturing Element Integration & Ground Test Flight Testing & Verification Increasing Opportunity For Block 06 Block 06 Design & Development Manufacturing Element Integration & Ground Test Flight Testing & Verification Maximum Opportunity For Block 08 Block 08 Design & Development Manufacturing Element Integration & Ground Test Flight Testing & Verification Time

12. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

13. MDA ASSURANCE REPRESENTATIVES MARs provide MDA with deployed Mission Assurance resources for direct and unfiltered supplier insight, with the leverage to effect value added change and mitigate risk in a timely fashion, and serve as MDA’s Mission Assurance ambassadors to its Supplier community, associated agencies and organizations Responsibilities: • Rapidly identify, report and resolve technical issues on both hardware and software that impact the safety and reliability of the Missile Defense program • Immediately report to the supplier’s program office and to MDA/QS any safety concerns/issues • Immediately report to MDA/QS significant hardware/software issues impacting upcoming missions • Report and resolve safety, quality, and reliability issues/concerns impacting MDA hardware and software

14. MDA ASSURANCE REPRESENTATIVES (CONT’D) • Responsibilities (cont’d): • Represent QS Directorate and Cognizant Program Office • Provides DCMA and contractor direct access to MDA • Provides MDA Director unfiltered insight into program operations in real-time • Participates in design integration, test, qualification activities and issue resolution • Ensures that all applicable safety, quality and mission assurance provisions are being followed • Disseminates lessons learned to counterparts at all BMDS components and DCMA • On-site authority for QS LOD to DCMA

15. MAR vs DCMA ROLES DCMA MARs

16. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

17. Unfettered Access Direction

18. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

19. Product Design • Product Requirements • Processes “Comparison” • SQMA Requirements • MDA/Industry Best-Practices (MDA Assurance Provisions & Other MIL-STDs/Guidelines) • Contractor Plans, Procedures, Practices and Records AUDIT PROCESS MDA ASSURANCE AUDITS 19

20. MDA ASSURANCE AUDITS • Validate that end items at a selected level conform to engineering design requirements • Validate end item at a selected level is qualified • Compare fabrication methods and acceptance test levels against MDA & industry standards as appropriate for MDA use • Validate production readiness • Assess Contractor/Government Mission Assurance practices, procedures, and implementation

21. MISSION ASSURANCE AUDIT SCOPE • Management Systems • System Design • Integrated Test and Evaluation • Supplier Management • Configuration Management • Safety • Reliability, Maintainability, and Availability • Software • Parts, Materials, and Processes • Fabrication and Quality • Quality Management System – AS9100 • Other Safety, Quality, & Mission Assurance Practices

22. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

23. MDA ASSURANCE GROUP The MDA Assurance Group (MAG) is used to ensure industry-wide understanding and implementation of the MDA Quality, Mission Assurance and Safety requirements, as set forth by the MDA/QS Director MAG membership includes key Government and aerospace industry executives knowledgeable of quality enterprise strategic planning, Quality Management System (QMS) requirements / implementation, mission assurance principles, and knowledge of their companies Missile Defense programs

24. MDA ASSURANCE GROUP (CONT’D) • Primary responsibilities of the MAG are to: - Review and discuss the MDA Quality, Mission Assurance and Safety policies and strategies set forth by the MDA Director to make certain implementation facilitates mission success • Identify, discuss and resolve significant, cross-cutting systemic issues affecting MDA programs • Understand the activities of other Aerospace Industry groups/associations like SQIC, AAGQ, AIA, etc with purpose to eliminate redundant activity and to gather their lessons learned • MAG Meetings were held on 15 March 2005 and 28 February 2006 • Key MAG Members: MDA, NASA, Boeing, Aerojet, Northrop Grumman, Raytheon, Lockheed Martin, ATK • Ex: Spawned “Stakeholder Initiatives”

25. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

26. MDA ASSURANCE ADVISORIES • MDA Assurance Advisories (MAA) provide information about problems, potential problems and failure experience data on parts, components, materials, manufacturing processes, specifications, computer software, test equipment and safety • Advisories are exchanged to improve quality, reliability, and delivery schedules and to reduce costs • Implemented in October 2003, QS has published 23 Advisories. Distribution to the MDA program elements, Industry, NASA, NRO and SMC

27. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

28. DCMA DELEGATION OF AUTHORITY • Agreement between the Missile Defense Agency Director and the Defense Contract Management Agency Director… • Establishes quality oversight requirements for mission critical items • Defines MDA expectations and priorities • Empowers DCMA to provide hands on quality insight/oversight • Establishes quality assurance insight/oversight requirements throughout the supply chain • Provides commonality and consistency for quality assurance implementation • Provides a government influence on mission success

29. DELEGATION PROVIDES… • DCMA Quality Assurance oversight requirements for: • Quality assurance surveillance strategy & plan • Process surveillance activities • Contract management offices reporting and notification • Material review board activities • Mandatory Government Inspections (MGIs): • Critical lifts and operations • Test setup (i.e., equipment and crew certifications, documentation verification) • Final inspection • Pre-shipment and shipment activities (i.e., black box closeout, DD250, etc.)

30. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

31. PARTS, MATERIALS AND PROCESSES PROGRAM • State of MDA PMP Today • Non-standardized PMP management • COTS Technology and Obsolescence management lacks definition • MDA lacks a comprehensive product engineering process • MDA/QS PMP • MDA-wide PMP Management Plan • MDA Parts, Materials and Process Board/Control Boards • MDA Preferred Parts List • Pilot Program developed for GMD Product Lines

32. Why the PMAP? • PMAP meets two of the Director’s goals: • Support the operations and sustainment of capabilities fielded to the warfighter • Eliminates variability in screening and qualification of PMP • Improves the probability of mission success • Ensures repeatability in the qualification baseline • Develop an integrated capability for Block 2006 and beyond • Through a fully integrated Agency PMP program Uniformity of Requirements Across All Elements

33. Why the PMAP is Necessary…Industry State of Affairs • Overseas Plants • Limited processes and material change knowledge • Military parts built on comm. manufacturing lines • Easy access: foreign countries reverse engineer military technology • Unauthorized Distributors • No established relationship with part vendors, poor quality, no recourse – traceability issues • Inconsistency in vendor lot to lot parameters • Counterfeit fasteners - rivets, nuts, bolts, washers • Lead-free Initiative • Pandemic for Hi-Rel systems • No accepted method for tin-whisker mitigation • Government - military and space application • Obsolete Parts • EEE parts represent decreasing market share Commercial Driven Industry = Greater Controls Required

34. Why a PMP Program is Necessary… Counterfeit Components • Unauthorized Manufacturers • Build fake parts, selling cheap or defective devices at brand name prices • Buy low-power die for high-power packages • Parts marked with “high-rel” part no. • Fabricate parts to substandard processes • China # 1 Seller • $1 - 10 billion per year (estimate) • 35% of world market by 2010 • Fab sites expanding rapidly, little govt control • Patent infringement • Companies Counterfeited • Motorola - On Semi • Toshiba - National • PMC-Sierra Counterfeit Genuine Counterfeit capacitors with substandard electrolyte Counterfeit Genuine Note the difference in part numbers

35. PMAP Requirements & Section Highlights • Introduction Scope - Purpose • Applicable Documents Government, Commercial • PMP Management Structure, Roles, and Responsibilities PMP Control Board • PMP Requirements Program categories and definitions Management of PMP selection & Quality Requirements Requirements Applicability Matrix PMP Procurement Rad Hard, GFE, GFM PMP Qualification Non-Conforming Materials and Parts Failure Analysis ESD Control, Environmental Controls, Handling Controls Preservation, Packaging and Storage Requirements • Appendices Derating PEMS COTS Radiation Hardness Prohibited Parts & Materials

36. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDa Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Software Acquisition Process Improvement MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

37. SUPPLIER ROADMAP • Supplier Roadmap and Supply Chain Database • 453 Suppliers in 39 states and 2 foreign countries • Can be searched by: • Congressional District • State • Supplier • Program

38. SUPPLIER ROADMAP (CONT’D)

39. MDA/QS Director Quality, Safety And Mission Assurance (QS) Quality (QSQ) Safety (QSS) Mission Assurance (QSA) Assurance Integration (QSI) QS provides a continuous assessment of the engineering, technology, processes, production and programmatic practices applied to the complex and critical Ballistic Missile Defense System. MDA Assurance Provisions (MAP) DCMA Letter of Delegation MDA Assurance Representatives System/Range Safety Unfettered Access Parts, Materials and Processes Program Mission Assurance and Safety Audits Govt/Industry Lessons Learned Program MDA Assurance Group Test/Operational Safety (BSOs) MDA Advisory Process Supplier Roadmap

40. MDA LESSONS LEARNED ROADSHOW “The frustrating part about it is we have not had an incident that, in my view, can be classified as a failure of the technology. . . Someone didn’t perform a process right, somebody left out a piece of material where they shouldn’t. . .” • Former MDA Director, Lt Gen Ronald Kadish, December 2003 • Roadshow Targets all audiences: Program Offices, President, Vice President, Director, Chief Engineer, Systems Engineering Manager, IPT Leads, Engineers, Quality, Safety and Mission Assurance Personnel and Technicians • Discusses Lessons Learned on IFT-13c and IFT-14 Tests

41. Why Quality, Safety and Mission Assurance? Test Root Cause Fundamental Issue Inadequate laser firing unit qualification test. Stiffening foam was removed from a board. The random vibration test was conducted without power-on so the problem could not be detected on the ground. EKV failed to separate due to an ASIC lead fatigue failure in the laser firing unit on the EKV Test 1 Roundoff error propagation caused a timing mismatch. When the SIGI and FC clocks crossover, timing jitter occurs and occasionally data is sent to the ECU when it is busy, resulting in no responses. To avoid a recurrence, need to raise the tolerance level for missed messages -- changing one line of computer software code. Countdown automatically aborted due to a built-in-test anomaly that detected excessive drops of Electronic Control Unit (ECU) messages Test 2 One of the last steps in the launch sequence is to open the silo doors and retract the horizontal stabilizers that hold the missile in place. Sensors indicated that one of the three lateral support group mechanisms had not retracted, causing the missile to abort launch. Corrosion and dissimilar metals (galvanic corrosion) caused the failure. One mechanism failed to latch due to wrong crush blocks. Test 3

42. MDA LESSONS LEARNED ROADSHOW (CONT’D) CompanyGroupLocation Northrop Grumman KEI Fair Lakes, VA Northrop Grumman MA Leads Reston, VA Lockheed Martin Aegis Moorestown, NJ Raytheon RMS Woburn, MA Raytheon IDS Tewksbury, MA THAAD Program Office Huntsville, AL Lockheed Martin THAAD & MKV Sunnyvale, CA Lockheed Martin THAAD Huntsville, AL Lockheed Martin BV+ Courtland Elkton, MD AF SPO ABL Albuquerque, NM Raytheon RMS Tucson, AZ Spectrum Astro NFIRE, STSS Gilbert, AZ Northrop Grumman STSS Los Angeles, CA Aerojet MDA Sacramento, CA

43. Getting Back to the Basics WHAT MDA HAS DONE TO INCREASE SQMA • Increasing Quality Assurance rigor by: • Organizing for Mission Success • Changing Culture • Onsite MDA Mission Assurance Representatives (MARS) • MDA Assurance Provisions (MAP) & PMAP • Defense Contract Management Agency (DCMA) Delegation of Authority Letter • System and Range Safety Oversight • Safety and Occupational Health

44. GOING FORWARD • Change Customer and Contractor Cultures • Corporate Leadership and UNIVERSAL, CONTINUING Commitment • Increase Process Discipline – Supplier Management • Management and Engineering Process Rigor • Establish and Maintain Basic Set of SYSTEM Standards (MDA MAP – Core Standards) • Ensure basics are CONTINUOUSLY accomplished by BOTH Customer and ALL Supplier Tiers • Ensure Systems Engineering, Disciplined Test Management, Quality Management, Risk Management, Configuration Management • Organize for Mission Success • Clear, continuing organizational accountability • Robust engineering organization • Independent, capable SQMA organization and personnel