Value-Added Aerospace Auditing Workshop | R. Darrell Taylor | AS9100 QMS

1. WELCOME! Americas Aerospace Quality Group AAQG Registration Management Committee RMC Auditor Training and Workshop Session Minneapolis, MN July 30-31, 2009 Value Added Aerospace Auditing

2. Who is R. Darrell Taylor? • Raytheon Corporate AS9100 QMS Auditor • 27 years experience in the quality field (inspector, engineer, teacher, director) • RABQSA certified Lead QMS auditor, AIEA • ASQ – CQA, CMQ/OE, CQPA • 6 s Master Black Belt, certified Black Belt, Raytheon 6 s Specialist

3. Root Cause & Corrective Action Analysis Training – Why Needed? At a recent IAAR (Independent Association of Accredited Registrars) meeting the question was asked, why are OEMs, ABs and ICOP all focused on correction, root cause analysis and corrective action? Simply stated, CBs and suppliers are not delivering to the Aviation, Space and Defense (aerospace) industries expectations! It is essential all CBs and auditors support industry in its quest to improve these activities within the CB community, its auditor base and throughout the supply chain. We can make a difference! Reg Blake – Primary IAAR Rep to AAQG RMC and Americas CB Rep to IAQG OPMT

4. Root Cause & Corrective Action Analysis The scope of this workshop is to explore how we, as auditors, can analyze auditee corrective action and root cause for accuracy, relevancy, and completeness for the purpose of improving the auditee’s system and have a positive impact on the aerospace industry.

5. “It is not necessary to change.Survival is not mandatory.” Quote from : Dr. William Edwards Deming

6. What is the Auditors Role? • Identify Requirements • Verify Conformance to the Requirements • Verify the effectiveness of the process

7. Topics • Process • Corrective Action Initiation • Root Cause Analysis • Corrective Action • Verification & Validation

8. Questions to Explore What constitutes real root cause analysis? How is root cause analysis documented? Who should be involved? How do we determine if corrective action is effective? Who should do this?

9. AS9100 -8.5.2 Corrective Action • The organization shall take action to eliminate the causes of nonconformities in order to prevent recurrence. Corrective actions shall be appropriate to the effects of the nonconformities encountered. A documented procedure shall be established to define requirements for • a) reviewing nonconformities (including customer complaints), • b) determining the causes of nonconformities, • c) evaluating the need for action to ensure that nonconformities do not recur, • d) determining and implementing action needed, • e) records of the results of action taken (see 4.2.4), • f) reviewing the effectiveness of the corrective action taken, • g) flowing down corrective action requirements to a supplier when it is determined that the supplier is responsible for the nonconformity, • h) specific actions where timely and/or effective corrective actions are not achieved, and • i) determining if additional nonconforming product exists based on the causes of the nonconformities and • taking further action when required.

10. Approve Corrective Action Containment Implement Corrective Action Identify a Nonconformity Root Cause Initiate Corrective Action Corrective Action Process Conduct Verification & Validation What We Control What the Auditee Controls

11. What do you like?What can be done better?

12. Sample Finding - Observation Despite the procedures in place, a certain amount of undocumented equipment swapping and borrowing occurs. Training should be conducted to ensure that equipment such swapping/borrowing is approved by the assigned owner of the equipment. Otherwise, there is the danger of equipment not being available to the owner when needed, since verbal agreements, as demonstrated above, are easily forgotten. Convincing evidence was given showing that there was no chance of an Out of Tolerance item being used for testing, which is why this is an Observation and not a finding. Further findings of this sort would lead to a minor nonconformance.

13. Corrective Action Initiation NOTE: Use AS9101 form or equivalent

14. Definitions • ASQ (The Quality Audit Handbook) • Finding - A conclusion of importance based on observation (s). • Audit Standards (Willborn) • Audit Findings – are the significant conclusions and information needed by the client or audit user for attaining the planned audit objectives.

15. Problem Definition - Simple Approach • Four Elements: • Source of the Requirement • Requirement • Source of the Finding • Finding Fundamentals of Quality Auditing

16. Requirement Requirement Source Nonconformance Nonconformance Source Example AS9100, paragraph 8.2.2 “Internal Audit” states “The organization shall conduct audits at planned intervals…” At XYZ company there is no audit plan.

17. Problem Definition • 4W: • What was affected • Where did the problem take place • When was the problem discovered • Who Discovered the problem • 2H: • How much was affected • How often has the problem occurred • 1C: • What is the consequence Correct, Prevent, Improve

18. Complete Problem Definition • What is the problem? • When did it happen? • Where did it happen? • What is the significance of the problem? Apollo Root Cause Analysis

19. Corrective Action Initiation Requirements • Write complete sentences • The problem description needs to be written clearly • Stick to the facts. Don’t make it a story. • Be precise in defining the requirement and deficiency • Avoid abbreviations, acronyms… • Noncompliance issues (Major, Minor) need to “violate” some requirement or they may document the absence of a process • Program plan, procedure, standard (AS9100), etc. • Always include the violated requirement in the problem description

20. Problem Description Requirements (cont.) • Problem Description Sufficiency • Identify What, Where, When, as appropriate • Reference the Evidence • The responsible assignee needs to have enough information from the problem statement to conduct root cause analysis and identify corrective action • Self-check the problem description • Is the statement complete and self-sufficient? • Ask yourself … So What? • Where does it say that I have to?

21. “A problem well defined is a problem half solved” (anonymous)

22. CASE STUDY 1 WRITING A PROBLEM DESCRIPTION

23. Levels of Nonconformities

24. AS9101 - DRAFT Major Minor NOTE: An observation is not a nonconformity

25. AS9101 - DRAFT • 3.2 Major Nonconformity • A non-fulfillment of a requirement which, is likely to result in the failure of the quality management system or reduce its ability to assure controlled processes or products; it can be one or more of the following: • a nonconformity where the effect is judged to be detrimental to the integrity of the product, or service; • the absence of, or total breakdown of a system to meet a 9100-series standard requirement, an organization procedure, or customer quality management system requirement; • any nonconformity that would result in the probable shipment of nonconforming product; and • a condition that could result in the failure or reduce the usability of the product or service and its intended purpose.

26. AS9101 - DRAFT • 3.3 Minor Nonconformity • A non-fulfillment of a requirement which is not likely to result in the failure of the quality management system or reduce its ability to assure controlled processes or products; it can be one of the following: • a single system failure or lapse in conformance with a 9100-series standard or customer quality management system requirement; or • a single system failure or lapse in conformance with a procedure associated to the organization's quality management system. • NOTE: A number of minor nonconformities against one requirement, e.g. on different sites, can represent a total breakdown of the system, and thus be considered a major nonconformity.

27. What do you like?What can be done better?

28. EXAMPLE • Nonconformance Description: • Some departments are not included in the distribution list of relevant quality procedures such as Engineering was not included on the distribution list for Procedure XYZ, Control of Nonconforming Material, although Engineering is involved in the disposition decisions. • Root Cause: • Inadequate review of procedure needs. • Corrective Action: • Company A will issue Procedure XYZ to Engineering by 12/31/200X. • Company A will then post Quality Procedures on its intranet with access available to all departments.

29. Containment Action • Containment addresses the immediate issue. • For example, you have a car that will not start. Containment gets it started. • If we have a controlled process to keep measuring equipment calibrated and someone finds un-calibrated or delinquent calibration dates, the containment would be to get the equipment calibrated. • During development of Containment, the auditee will determine how big is this issue. Stop the bleeding!!!

30. Root Cause Analysis

31. Root Cause • DEFINITIONS: • Root Cause (s): The original event (s), action (s), and/or condition (s) generating (directly or in cascade) an actual or potential undesirable condition, situation, nonconformity or failure.* • Note : There are often several root causes for one problem* • Contributing Causes: Contributing causes are causes that taken alone would not cause the problem but can increase the risk of the issue to happen. Analysis for these causes generally require taking small steps (or a finer look) to be identified and fixed.* • Root Cause Analysis: The process of identifying all the causes (root causes and contributing causes) that have or may have generated an undesirable condition, situation, nonconformity or failure.* • Examples of traditional root causes tend to focus on lack of, or currency of a procedure, training of stakeholders in applicable policy/procedures, lack of an effective tool to perform a task, etc. *IAQG - SCMH

32. Requirement Requirement Source Nonconformance Nonconformance Source Example • AS9100, paragraph 8.2.2 “Internal Audit” states “The organization shall conduct audits at planned intervals…” • At XYZ company there is no audit plan.

33. Root Cause Analysis Process Capability Barrier Analysis 5 Why’s Control Plans Pareto Charts Fishbone Tree Diagrams Process Maps Risk Tree Analysis Apollo Pie Charts Interrelationship Digraph Run Charts Histograms DMAIC Change Analysis 5 Why’s Process FMEA Brainstorming Affinity Diagrams Flowcharts DOE Design FMEA Control Charts Events & Causal Analysis Scatter Diagram Force Field Analysis

34. Response • Containment – N/A • Root Cause Analysis – Did not understand the requirement • Corrective Action – Write plan COMMENTS?

35. Why do we have poor root cause analysis?

36. Reasons – We have not set criteria about what makes an acceptable corrective action plan (we are satisfied when we don’t receive any more defective parts and stop here) – We continue to accept bad answers (even if say we will not, reality of life makes us accepting ) – People (internal and external) have not the Root Cause Analysis culture, don’t know any process or are not effectively trained *IAQG - SCMH

37. Investigation Checklist • Describe the fast fix that was taken, Is it working? • Identify owner and those involved. • Gather information: • Data • Employee Input • Flowcharts of the process • Procedures • Records – (quantitative data) • Has the problem occurred in the past? • Identify Root Cause. Book: Correct! Prevent! Improve!

38. Rules of Investigation • Use proven root cause analysis tools • Think “out – of – the – box” • Take the time needed • Put a plan together Book: Correct! Prevent! Improve!

39. Actions and Conditions Match Strike Match Oily Rags Oxygen TIME Apollo: Root Cause Analysis

40. Apollo Action • There is no such thing as a single root cause!! Condition Action Condition Action STOP Beyond your area of control Condition Condition Condition Action Condition Condition Condition

41. 5 WHY’s APOLLO OBSERVATION MINOR MAJOR Root Cause Requirement? Understand the severity of the problem MANDATORY RECOMMENDED IMPACT OPTIONAL NOT REQUIRED FREQUENCY Modified from IAQG - SCMH

42. What Makes a Good Root Cause? • Document the primary cause (s) of the nonconformance • Understand the problem statement • Don’t assume that people know or understand the problem well enough to determine root cause • Data collection and analysis • Involve people who have an understanding of the problem • Use the cause and effect principle • Facilitate the Root Cause Analysis • Provide the knowledge of RCA tools • When solved, the problem will not recur

43. Analyzing a Root Cause • How does an auditor evaluate a root cause? • CONNECT THE DOTS!! • Does the root cause fit the problem statement? • Does the root cause fit the containment? • Is there any indication of the use of “tools”? • Is this a repetitive root cause? • Is this a point analysis, or a system analysis?

44. What do you like?What can be done better?

45. Example • Corrective Action Subject • Residue on contact causing intermittent electrical connection. • Problem Description: • Found 2 units of XYZ which has intermittent electrical connection. Contact pieces are found with glue residue. This is non- conforming to note 3 of assembly drawing XYZ rev G.

46. Root Cause Analysis • After investigation it was discovered that the adhesive permabond did not have a regulating component to restrict the amount of flow that was being applied. This caused an overflow of glue into the area and the timing of the subsequent assembly did not allow for the detection of the contamination. None of the documentation identified the need for this regulation device. This was indicated during the initial build however the documentation did not reflect what was considered a common practice.

47. Corrective Action Short Term (immediate action for existing material) • Amend work instructions to emphasize that board and domes must be thoroughly cleaned and domes are not to be installed for eight hours after board installation. • Corrective Action Long Term (long term preventive action) • The bottle that contains the adhesive was modified with a regulator to control the flow of the glue and to limit the expansion of the glue once pressure is added. Also the Immediate Corrective Action will be a secondary action applied to further safe guard

48. Evaluating Corrective Actions Definition – Action taken to eliminate the causes of an existing nonconformity, defect, or other undesirable situation in order to prevent recurrence – ASQ Audit Handbook

49. Corrective Action • This is the specific action (s) you take to address the root cause of a problem. • They will only be effective if you have identified the true root cause of the problem. • For example, cleaning battery terminals or replacing a battery in your car will not resolve the problem of a car that will not start if the root cause of the problem is a bad alternator. • Corrective action includes the changes in policy, procedure and training to affected stakeholders with assignments to specific individuals and scheduled dates for task completion.

50. Elements of a Corrective Action • Containment – Stop the Bleeding • Root Cause Analysis • Design, Implement Corrective Action Plan • Verify Corrective Action Plan is Implemented • Validate the effectiveness of the Corrective Action Plan