Space Forum RMC Presentation June 21-22, 2007

1. AS9100 Impacts On Mission Assurance Space Forum RMC Presentation June 21-22, 2007

2. Drivers in the Space Industry How Does This Uniqueness Impact the Product Delivery Model How Can it Impact Suppliers Business Model Unique AS 9100 Interpretation Needs Auditor Discussions on Considerations in auditing a supplier to the Space Industry How can we better support/service the industry Address performance issues that inherently impact the industry Presentation Focus

3. Unique Products in Small Quantities Rapid Application of Technology into production systems Product Not Returnable (in most cases) Unique Operational Environments Event Driven business-Only as good as the last mission Events impact the Industry (NRO, NASA, MDA,SMC,DOD) Risk not fully understood in supply chain Pushing the margins requires perfection Drives A Need for Mission Assurance in the Supply Chain Drivers in the Space Industry

4. Importance of AS9100 for Space Products • 100% mission success is critical to customers and our nation • The AS9100 QMS ensures product performance for success • Limited opportunity to rework/repair after delivery • Space systems are typically highly complex and low production • Identification of Key Characteristics plays an important role to mitigate risk of non-conformance escape or late detection • Detect non-conformance as early as possible • Space systems designs drive the use of emerging technologies • Ensure new technologies are integrated with appropriate management controls • Mitigate negative impact on product

5. Importance of AS9100 for Space Products • Ensure customer’s desires become reality • Confidence that AS9100 certification provides assurance of qualified QMS. • AS9100 compliance is a key factor in Supplier selection • Supply Chain management is a critical area of management concerns

6. Identification of Mission Assurance Concepts Risk Key Characteristics/Critical Requirements Mission Assurance Application to a QMS Relationship to AS9100 Text Subject Matter Discussions

7. Risk Criteria / Requirements • Program plans • Contractual requirements • AS9100 • Mission Assurance Plan (MAP) • Certificate of Flight Readiness (CoFR) • Reliability program requirements Space Factors / Unique • Unique skills and processes • Large system integration • Unique operating environments • Limited recall (ISS on orbit) one of a kind products • Critical items control / management • Human factors skill / training • Processing induced hazards Audit Focus • Elevation of risk (communication) • Defining of risks and control • Mitigation • Abatement • Authority to accept • System/structure • Accountability • Knowledge of improper technique and impact • Won’t fail or process proofing Examples • Independency not maintained on eliminating risks—Challenger O –Rings focused on understanding and testing • Process change impacting material properties • Human factor examples • Proper installation • Understanding of process • Skill and proficiency

8. What is Risk? A risk is a potential future event that could result in adverse and unplanned consequences. All Risks contain the following three elements: • A risk is a future event. • The likelihood of the future event occurring is greater than 0%, but less than 100%. • The consequence of the future event will be adverse, unfavorable, or negative. A risk is NOT a Problem, an Issue or a Crisis! Riskis a measure of the potential inability to achieve overall program objectives within defined cost, schedule and technical constraints. (Reference: Risk Mgt Guide for DoD Acquisition, 4th Edition, June 2003) Theremust bea definition of what the organization considers risk Basic Requirementto look for when auditing.

9. The Risk Management Process Plan entire strategy Choose an approach Anticipate what can go wrong Identify Plan Decide what is important Assess Continuously communicate Communicate Handle Determine necessary actions Monitor Correct deviations Track results

10. Risk Management Steps • Risk Planning • The step of developing and documenting comprehensive and interactive strategies and methods for identifying and tracking risk areas, training, developing risk mitigation plans, performing risk assessments to determine how risks have changed, and planning/obtaining adequate resources. • Risk Identification • The step of discovering and defining all risks inherent in your program or project. • Risk Assessment • The process of analyzing and prioritizing program and process risks against cost, schedule and/or performance criteria. • Risk Handling • The step that identifies, evaluates, selects, and implements actions in order to reduce risk likelihood or consequence to an acceptable level. • Risk Monitoring • The step that systematically tracks and evaluates the performance of Risk Handling actions against established metrics throughout the acquisition process.

11. Risk Management Steps Risk Planning, The organization or program should define their terms, show understanding of their objectives and goals, show understanding of the organizational culture and develop/use methodologies and templates. Audit Evaluation: Ensure that some type of risk management planning has been accomplished and is being followed.

12. Risk Management Steps Risk Identification Is the process of identifying risks in the program and critical processes that could negatively affect meeting functional, schedule and cost objectives. Identifying program risks is the most important step in the Risk Management process. The caliber and quality of the identification effort establishes the effectiveness of the entire Risk Management effort. Audit Evaluation: Ensure that there is a specific format for the Risk Identification Statement and that the format is followed.

13. Risk Management Steps Risk Assessment: Once risks have been identified, to your program, an organization or program must determine the likelihood that these risk events will occur and, if they do occur, their consequence. Assigning Likelihood and Consequence is the purpose of Risk Assessment. Audit Evaluation: Is there a methodology to assign likelihood and consequence to each risk and is it followed?

14. Risk Management Steps Risk Handling: To identify, evaluate, select, and implement Risk Handling options to set risk at acceptable levels, given program constraints and objectives. Risk Avoidance- Avoiding the risk is removing the root cause. An example is changing the requirements to a level that lowers risk to an acceptable level, but still meets the user's needs. Risk Transfer - Transferring the risk is moving it to another entity to achieve an overall reduction in Program risk. An example is reallocating design requirements to those system elements that can achieve the system requirements at a lower risk level. Risk Control - Risk Control is the most widely used Risk Handling technique. Risk Control is taking active steps to minimize the risk’s impact on the program, either by reducing the likelihood of its occurrence or by minimizing the consequence. Risk Assumption - Assuming the risk is accepting the risk without engaging in any special effort to mitigate, manage, plan for, insure against or otherwise address the risk Audit Evaluation: Are risk handling concepts being assessed and carried out for each risk?

15. Risk Management Steps Risk Monitoring Is a continuous process that systematically tracks and evaluates the performance of Risk Handling actions against established metrics. It should continue throughout the Program Life Cycle. Audit Evaluation: Is a risk monitoring process established and is it being followed?

16. Application of Risk to Key Management System Processes Risk Management Risk 7.2.2 Others ??? Requirements Development 7.1 & 7.2 Design Control 7.3 Purchasing 7.4 Supplier Oversight 7.4 Configuration Management 4.3 Inspection & Test 8.2.4 How Do the Words of AS 9100 Change when Risk is Applied to them???

17. 4.1 General Requirements: The organization shall establish, document, implement and maintain a quality management system and continually improve its effectiveness in accordance with the requirements of this International Standard. The organization shall a) identify the processes needed for the quality management system and their application throughout the organization (see 1.2), b) determine the sequence and interaction of these processes, c) determine criteria and methods needed to ensure that both the operation and control of these processes are effective, d) ensure the availability of resources and information necessary to support the operation and monitoring of these processes, e) monitor, measure and analyse these processes, and f) implement actions necessary to achieve planned results and continual improvement of these processes. Where an organization chooses to outsource any process that affects product conformity with requirements, the organization shall ensure control over such processes. Control of such outsourced processes shall be identified within the quality management system. 5.4.2 Quality Management System Planning: Top management shall ensure that the planning of the quality management system is carried out in order to meet the requirements given in 4.1, as well as the quality objectives 4.1 similar to steps of Risk Management Risk Planning (4.1.b, 5.4.2) Risk Identification (4.1.a, 6.1, 7.1) Risk Assessment (4.1.c, 6.1, 7.1) Risk Handling (4.1.d, 6.1, 7.1) Risk Monitoring (4.1.e, 8.1, 8.2.3, 8.2.4) Risk Management – AS9100 Impacts –

18. 6.1 Provision of Resources: The organization shall determine and provide the resources needed a) to implement and maintain the quality management system and continually improve its effectiveness, and b) to enhance customer satisfaction by meeting customer What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk Management – AS9100 Impacts –

19. 7.1 Planning of Product Realization: The organization shall plan and develop the processes needed for product realization. Planning of product realization shall be consistent with the requirements of the other processes of the quality management system (see 4.1). In planning product realization, the organization shall determine the following, as appropriate: a) quality objectives and requirements for the product; b) the need to establish processes, documents, and provide resources specific to the product; c) required verification, validation, monitoring, inspection and test activities specific to the product and the criteria for product acceptance; What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk Management – AS9100 Impacts –

20. 8.1 General: The organization shall plan and implement the monitoring, measurement, analysis and improvement processes needed a) to demonstrate conformity of the product, b) to ensure conformity of the quality management system, and c) to continually improve the effectiveness of the quality management system. This shall include determination of applicable methods, including statistical techniques and the extent of their use. 8.2.3 Monitoring and Measurement of Processes: The organization shall apply suitable methods for monitoring and, where applicable, measurement of the quality management system processes. These methods shall demonstrate the ability of the processes to achieve planned results. When planned results are not achieved, correction and corrective action shall be taken, as appropriate, to ensure conformity of the product. What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk Management – AS9100 Impacts –

21. Key Characteristics/Critical Requirements Criteria / Requirements • MAP (MDA document) • AS9100 section 7 • Program plans • NSTS 07700 (NASA requirements) • FMEA / CIL Space Factors / Unique • Definition of key characteristics • FMEA CIL application • Change control • Flowdown to manufacturing documents • Requirements for second-set-of-eyes Audit Focus • Rigor in management and control • People understanding Key features-actively managed • Reliability Focus • Change management based on history (IFA / failure) • Escape management • Violation of material and CSP procedures • Personal understanding - witness, verify, & measure • Visibility of waivers/noncompliance-asses risk Examples • Critical feature not identified / part fails in test • Critical Safety Items • Wrong filters installed - restrictive flow in test • Material certs not completed • Material coupons not matching parts • Cost trade-offs introduces risk

22. What do we think of when we think of Requirements???? Requirements • Customer • Contractual • Program/Project • Design • Form, Fit, Function • Intended Use

23. Risk Stated in Section 7.2 Is Risk Implied in other Sections of AS9100?? Prioritization – How does this fit with Risk??? Determination of Importance Consequence & Likelihood Etc. Prioritization – How does this fit with Requirements??? Customer/User Needs Product Needs Process/Project Needs Focus of Efforts Critical Requirements Key Characteristics Risk Associated With Requirements

24. What Do We Mean By Critical Requirements Mission Mission Requirements Must All Be Met But Are They All Equal In Significance? • Mission Requirements • Functional Objectives • Mission Environments CSI Non CSI • CSI – Critical Safety Item • System Requirements • Sub-System Req’ts • User Needs • Component Req’ts • Part Requirements • Criticality Determined By: • Risk Analysis • FMEA • Etc.

25. Key Characteristics/Critical Requirements • Communication & Understanding of Risks • Risk Based Decisions and Actions in Individual Processes • Operational Options that Need Risk Oriented Decisions associated with Critical Requirements • Design Approach • V&V Approach • Monitor & Insp. Approach • Supplier Oversight • Actual Determination of: • Requirements • Risk Prioritization 1st Use of Prioritization of Requirements -Key Characteristics- Current Use of Word Risk RFQ Proposal Contract Design Manufact. Integrate Product Delivery Monitoring and Inspection Activities Communication of Supplier Requirements -Key Characteristics- Purchasing Suppliers All Requirements are not created equal

26. Risk & Prioritization • Opinions on this Concept of Critical Requirement Control • Identification • Prioritization (Risk Based) • Communication • Decision Making (Risk Based) • Agree/Disagree • Why • Does The AS 9100 Standard Allow us to Audit This Concept

27. 7.1 Planning of Product Realization: The organization shall plan and develop the processes needed for product realization……. In planning product realization, the organization shall determine the following, as appropriate: a) quality objectives and requirements for the product; c) required verification, validation, monitoring, inspection and test activities specific to the product and the criteria for product acceptance; What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

28. 7.2.1 Determination of Requirements Related to the Product: The organization shall determine a) requirements specified by the customer, including the requirements for delivery and post-delivery activities, b) requirements not stated by the customer but necessary for specified or intended use, where known, d) any additional requirements determined by the organization. 7.2.2 Risk (e.g. new technology, short delivery time scale) have been evaluated What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

29. 7.3.1 Design and Development Planning: The organization shall plan and control the design and development of product. During the design and development planning, the organization shall determine b) the review, verification and validation that are appropriate to each design and development stage, and c) the responsibilities and authorities for design and development. Where appropriate, due to complexity, the organization shall give consideration to the following activities: - structuring the design effort into significant elements; - for each element, analyzing the tasks and the necessary resources for its design and development. This analysis shall consider an identified responsible person, design content, input data, planning constraints, and performance conditions. The input data specific to each element shall be reviewed to ensure consistency with requirements. Planning output shall be updated, as appropriate, as the design and development progresses. The different design and development tasks to be carried out shall be defined according to specified safety or functional objectives of the product in accordance with customer and/or regulatory authority requirements. What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

30. 7.3.2 Design and Development Inputs: Inputs relating to product requirements shall be determined and records maintained (see 4.2.4). These inputs shall include a) functional and performance requirements, d) other requirements essential for design and development. 7.3.3 Design and Development Outputs: The outputs of design and development shall be provided in a form that enables verification against the design and development input and shall be approved prior to release. Design and development outputs shall d) specify the characteristics of the product that are essential for its safe and proper use, and e) identify key characteristics, when applicable, in accordance with design or contract requirements. What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

31. 7.3.5 Design and Development Verification: Verification shall be performed in accordance with planned arrangements (see 7.3.1) to ensure that the design and development outputs have met the design and development input requirements. ………….. NOTE: Design and/or development verification may include activities such as: - performing alternative calculations, - comparing the new design with a similar proven design, if available, - undertaking tests and demonstrations, and - reviewing the design stage documents before release. 7.3.6 Design and Development Validation: Design and development validation shall be performed in accordance with planned arrangements (see 7.3.1) to ensure that the resulting product is capable of meeting the requirements for the specified application or intended use, where known. ……… NOTES: - Validation is normally performed under defined operating conditions. What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

32. 7.4.1 Purchasing Process: The organization shall ensure that purchased product conforms to specified purchase requirements. The type and extent of control applied to the supplier and the purchased product shall be dependent upon the effect of the purchased product on subsequent product realization or the final product………. What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

33. 7.5.1 Control of Production and Service Provision: Planning shall consider, as applicable, - the establishment of process controls and development of control plans where key characteristics have been identified, - the design, manufacture, and use of tooling so that variable measurements can be taken, particularly for key characteristics, and - special processes (see 7.5.2). The organization shall plan and carry out production and service provision under controlled conditions. Controlled conditions shall include, as applicable a) the availability of information that describes the characteristics of the product, c) the use of suitable equipment, d) the availability and use of monitoring and measuring devices, e) the implementation of monitoring and measurement, k) criteria for workmanship, which shall be stipulated in the clearest practical manner (e.g., written standards, representative samples or illustrations). What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –

34. 8.2.4 Monitoring and Measurement of Product: The organization shall monitor and measure the characteristics of the product to verify that product requirements have been met. This shall be carried out at appropriate stages of the product realization process in accordance with the planned arrangements (see 7.1). When key characteristics have been identified, they shall be monitored and controlled. 8.2.4.1 Inspection Documentation: Measurement requirements for product or service acceptance shall be documented. This documentation may be part of the production documentation, but shall include a) criteria for acceptance and/or rejection, b) where in the sequence measurement and testing operations are performed, c) a record of the measurement results, and d) type of measurement instruments required and any specific instructions associated with their use. Where required to demonstrate product qualification the organization shall ensure that records provide evidence that the product meets the defined requirements. What/Where Would We Audit?? Big Company Approach Small Company Approach What Questions Would We Ask?? What Type of Nonconformances (Observations) Would we Write Risk/Key Characteristics/Critical Requirements– AS9100 Impacts –