Lessons Learned and Corrective Action from IAQG’s RMC Oversight Assessment in Montreal

1. Lessons Learned and Corrective Action from IAQG’s RMC Oversight Assessment in Montreal Will Tate / Robert Flaharty Triumph Group / Spirit AeroSystems January 16, 2014 Other Party (OP) Assessor Workshop Palm Beach Gardens, Florida January 16, 2014

2. Objectives • Overview of the Americas Sector Audit conducted in Montreal, October 2013 • Findings • Opportunity for Improvements • Strengths • General Comments for AAQG • Feedback for IAQG OPMT

3. Oversight Requirement Complete an oversight of each SMS, as defined in the 9104/2 standard (AS9104/1, sec. 13.2a) Provide a mechanism for a periodic review of the lessons learned by each of the IAQG sectors. (AS9104/1, sec. 13.2b) Oversight of all entities of the ICOP (Industry Controlled Other Party) process is required as defined in 9104/1 and 9104/2 Aerospace Standards West Palm Beach, FL- January 16, 2014

4. SUMMARY OF OVERSIGHT • Overall the AAQG SMS Oversight was successfully completed as required • 1x Major and 4x Minor Non-conformances were raised • In addition, a further 16x Opportunity For Improvements (OFI’s) were identified

5. Summary of Findings • Minor – Delegation of responsibility for Auditor Re-Authentication • Minor – Timely closure of oversight NCR’s • Major – Correct application of the definition of Aerospace Audits to all AEA Re-authentication applicants • Minor – Oversight reports are not correctly documented or that records of oversight are maintained • Minor – Conflicts of interest are not assessed and effectively managed for oversight teams

6. AAQG # 01 (Minor): • It was not demonstrated that the currently documented arrangements effectively reflect the current arrangements for the control of the AAQG RMC auditor authentication activity as follows: • Auditor re-authentication review and decision-making are subcontracted from the RMC Auditor Review Committee to the respective AAB’s, RABQSA and INMETRO however the arrangement is not documented (ref: RMC Proc. 103) • Arrangements for the retention of auditor authentication or re-authentication records are not documented. • Procedure 103 was not demonstrated to be effectively implemented as the date of release was indicated as a draft •  In considering Procedure 103, the AAQG RMC may wish to review the division of responsibilities between the industry and the subcontract AAB to clearly document that the responsibility of auditor authentication and re-authentication decision-making is defined as being with either the RMC Auditor Review Committee voting members or with the subcontract AAB

7. AAQG # 01 (Minor): cont Corrective Action: The RMC 103 Procedure will be corrected to reflect the current revision’s release date by 1, Dec 2013 and a memo will be sent to the America’s AABs (INMETRO and Exemplar Global, formally RABQSA) defining the delegation of re-authentication approval of existing auditors by the auditor review team chair by 18 Dec 2013.

8. AAQG # 02 (Minor): • The AAQG RMC was not able to demonstrate the timely closure of oversight NCR’s in all cases (ref: 9104/2 Appendix J timescales) Corrective Action: • Update RMC Procedure 102 to include escalation process and more clearly define the roles and responsibilities of the process for escalation and oversight of the RMC for status of open issues. • With the release of AS9104-002 (2013) there will be a documented requirement for escalation after 90 days when issued during independent oversight in paragraph 7.15.2 b: for “shared” or supplemental. • The subject of closing and elevating NCRs over 90 days will be presented as a topic for future OP Assessor Training Workshops. • Oversight Chair will monitor for closure/status of open issues and present during closed RMC meetings. • ANAB documented an internal NCR in August for having 2 AB issued NCR’s (to CB’s) that went beyond 90 days and we did not move to suspend, as required.  ANAB’s corrective action was to modify our process and require a 75 day response from our CB’s on all AQMS minor NCR’s (All Major NCR’s are already at 60 days for response), this allows 2 weeks for any “back and forth” that may be needed.  This is documented in their newly released Accreditation Rule 29 (AR29).   Their EQM System workflow now flags its coordinators at day 75 and they in turn notify Bob Cruse of any that are nearing the 90 day mark so they can begin assembling an appeal panel to convene on (or as near as possible) the 90 day deadline.

9. AAQG#03 (Major): • It was not demonstrated that in all cases that RABQSA, as the AAB for the AAQG RMC, is correctly applying the definition of Aerospace Audits to all AEA Re-authentication applicants. • This finding is a recurrence of AAQG Oversight finding from May 2011. No evidence was available to demonstrate that corrective action from the May 2011 finding had been effectively implemented. Corrective Action: The RMC auditor review team will provide a follow-up review with Exemplar Global personnel on the correct type of audits to be included with auditor authentication and re-authentication documentation packages. The Auditor review team will perform periodic random checks of re-authenticating auditors, by requesting that their re-authentication documentation be forwarded to the auditor review team for review of their audit logs. The Auditor review team will request that AB auditors that have been recently re-authenticated, have their documentation reviewed by the auditor review sub team of at least 2 RMCAR members.

10. AAQG # 04 (Minor): • It was not demonstrated that in all cases oversight reports are correctly documented or that records of oversight are maintained and made available for oversight of the SMS. Corrective Action: • Review documents to verify if the records requirement is clearly stated and the location and process for storing is identified. • Communicate the records requirement at the upcoming OP Assessor workshop. Established a secure RMC website for records retention. Access limited to RMC voting members. • Review current process and update RMC procedure 102.

11. AAQG # 05 (Minor): • It was not demonstrated in all cases that conflicts of interest are assessed and effectively managed during the deployment of oversight teams conducting oversight on behalf of the AAQG RMC. Corrective Action: 1. Revise RMC procedure 102 to be clear on definition of "Conflict of Interest" as it relates to auditing AAB, CB, TPAB or AB. Procedure 102 to match requirements defined in to-be released AS9104-002. 2. Declare in advance to the auditee any potential conflicts and obtain permission to proceed prior to conducting the audit.

12. SUMMARY OF FINDINGS – OFI’s • OFI#1: Guidance for Travel Policy (for OP Assessor Oversight) is informally documented but may benefit from being formally covered in Procedure 102 Oversight or stand-alone procedure • OFI#2: AAQG RMC may benefit from maintaining a record of the approved AAB/TPAB procedures (with Rev status) for their approved AAB & TPAB subcontractors • OFI#3: Upon implementation of 9104-002 , the AAQG RMC may benefit from re-issuing current Form A & B’s to reflect any changes associated with implementation • OFI#4: AAQG RMC may benefit from updating Procedure 102 to reflect the current operating practice i.e. excluding CB, TPAB and AAB representatives from the Surveillance Sub Team • OFI#5: The AAQG RMC benefit from introducing a documented process to ensure that 9104-002 Form A’s (previously Annex I’s) are renewed after 3 years

13. SUMMARY OF FINDINGS – OFI’s • OFI#6: It was not demonstrated how the AAG RMC recognise stakeholders from other CBMC’s and IAQG Sectors e.g. AB’s, AQMS Auditors, TPAB’s, CB’s, TP’s and Training Courses • OFI#7: The recorded Objective evidence for CB Head Office ASA April 2013 was considered minimal and would benefit from including more detailed objective evidence • OFI#8: The AAQG RMC may benefit from maintaining a record of ICOP related implementation fees e.g. Auditor Authentication fees etc • OFI#9: Procedure 102 should clearly define the process for the management and escalation of NCR’s for independent OP Assessor Oversights e.g. OP Assessor contacts AB to escalate etc • OFI#10: Procedure 103 needs to reflect that RMC Auditor Review Committee makes a recommendation which is then endorsed by the full RMC

14. SUMMARY OF FINDINGS – OFI’s • OFI#11: Procedure 101 should include a statement to ensure ‘Rights of Access’ to all stakeholder data • OFI#12: The AAQG RMC should ensure that it maintains documented records of the decision making process (vote) for suspension or withdrawal of approval for a stakeholder e.g. AAB etc • OFI#13: AAQG RMC should consider introducing a Document Register that records all relevant scheme procedures and forms, including revision status • OFI#14: The AAQG RMC may benefit from increasing the duration for the 2013 Oversight to increase the effectiveness of the Office Oversight of INMETRO as the 2012 was 1 day covering both AB & AAB activity • OFI#15: The AAQG RMC may benefit from maintaining a list of attendees for OP Assessor Workshop Training courses.

15. SUMMARY OF FINDINGS – OFI’s • OFI#16: The current AAQG TP List published in OASIS, managed by RABQSA on behalf of AAQG RMC, contains a number of approved Training Courses which are not listed in 9104-003 e.g. ‘AQMS Foundation (Tailored)’ performed by SAI Global, AS9100 Standard Blended performed by Gladhill Associates International etc; the AAQG RMC may benefit from updating this List to only reflect approved Training Courses relevant to the ICOP scheme

16. STRENGTHS IDENTIFIED • Strong committed team of RMC members actively supporting the ICOP scheme in AAQG • Use of CB Report Cards to focus CB improvement • Strong working relationship with ANAB performing joint CB Oversights • Co-operation and assistance of the AAQG Auditee’s throughout this oversight with a genuine commitment to improve their processes.

17. GENERAL COMMENTS FOR AAQG • AAQG RMC must ensure they apply the lessons learned with RABQSA to future AAB/TPAB activities – ultimately the RMC is responsible for the performance of its subcontracted activities! • Consider performing an annual self SMS assessment using an independent OP Assessor • Beware of potential Conflicts of Interest and how you manage them effectively • For future AAQG Oversight, it is requested that the RMC ensures greater support from their stakeholders e.g. ANAB, RABQSA and INMETRO • It was identified that the 9104-001 Transition approval of INMETRO was not completed in accordance with SR002 timescales and suspension was not initiated in due time; however since the transition has now been completed, no NCR is being raised at this time.

18. General feedback for IAQG OPMT • Needs to be stronger in monitoring SMS Oversight NCR’s and ensuring these are closed in a timely manner • Must consider performing SMS Oversight in the week before the IAQG OPMT F2F meetings or as a totally separate activity • Consider having a more dedicated OP Assessor team to conduct the Oversights – this would ensure a consistent approach and might help to introduce common practices • SMS Oversight provides an excellent opportunity to learn from other SMS on different approaches, identify best practice etc • OASIS changes – functionality to be amended to show Suspension date for AAB/TPAB

19. General observations by the RMC team • Validate representation by all Americas required entities for the entire duration of the audit • RMC should establish a records administrator for RMC records management • Validate availability of all records 2 weeks prior to the audit date • Establish a minimum of 2 audit teams and designate a facilitator to track down items as required • Track closure of audit CAs as part of our Action itemstime log • RMC should conduct semi-annual internal audits (Jan/Jul) • Establish a timeframe for submitting completed oversight records for retention • Ensure all entities approved by the RMC have records retained that show evidence of the approval (specify where all records will be retained)

20. Robert Flaharty – Spirit Aerosystems Wichita, KS • 30 Years working in Supplier Quality • Supplier Audits, Source Inspection, Corrective Action • Systems Development • . • Will Tate – Triumph Group, Dallas, TX • 35 Years working in Aerospace & Defense • 15 Years Engineering, 20 Years QA and Process Improvement • Supplier QA, QMS, QA information systems • .