Joint Other Party Assessments

1. Joint Other Party Assessments ANAB’s Perspective Bob Cruse ANAB January 16, 2014 Other Party (OP) Assessor Workshop Palm Beach Gardens, Florida January 16, 2014

2. CB Office Assessments “Joint” Assessments with ANAB (AB)

3. Office Assessments…Planning • Acknowledge/confirm assessment assignment from ANAB coordinator (ANAB will then have confidence on assignment) • Communicatewith AB Team Leader (Travel logistics, etc...) • If you have not heard from the AB Team Leader, reach out to them, an email may have been missed/lost! • Ensure understanding of AB procedural requirements (Ref. AS9104/2 8.2.1)

4. Office Assessments…Planning • Don't make assumptions (take ownership of all travel details) • Don't come into the assessment with no knowledge of the CB's AQMS Certification levels/clientele. • This is available in OASIS.

5. Office Assessments…Planning • Don’t “come along for the ride” • Be prepared / knowledgeable / competent • An OP Assessor has a responsibility to oversight the CB, in addition to ANAB’s responsibilities • ANAB has not built in time to assist the OP Assessor through the entire process, certainly we are there for questions / confusion / caucusing…but not for training.

6. Office Assessments…Execution • Work with the AB • Be competent and confident in your ability • In order to fulfil 9104/2 requirements you must understand all that is being discussed in regard to AS. • You must be able to oversight on your own and review files, etc. to gain your own confidence in the system. (this oversight will supplement the AB’s assessment)

7. Office Assessments…Execution • If you have an NCR that the AB team agrees with please • write it clear and concise • include all the relevant information (Statement, evidence, requirement), • this will assist the AB team in the assessment closing meeting preparation.

8. Office Assessments…Execution • When the AB is assessing general requirements (e.g. internal audit, impartiality committee) and when the CB is explaining their certification system stay with the AB team to listen! • Don't venture off on your own too early, that will cause duplicated work and will cause a lack of understanding. Understand the system first! • Stay with the AB Assessor(s) until there is a mutually agreed upon AQMS objective that warrants you venturing off on your own.

9. Office Assessments…Execution • Don't spend so much time following an audit trail that due diligence cannot be given to additional files yet to be reviewed. Remember...Audits are samples, therefore, focus on what brings the most value to the audit objectives. • You can stop assessing when you find conformance or nonconformance…it is not your job to sort out why or how bad the problem is (that is the CB’s job).

10. Office Assessments…Execution • Don’t sit there like a bump on a log…you are NOT fulfilling your IAQG/AAQG member role if you do not have enough knowledge to oversee the CB. • Don’t expect the AB to hold your hand…we already have enough on our plate, we do not have time to train.

11. Office Assessments…Execution • Do not argue with the AB team! • If there is a disagreement that can not be resolved, provide the information back to the AB and the RMC Chair for resolution. • Do NOT ask the CB for a job! • You are there as an IAQG/AAQG member, NOT an OEM that may need a new/different opportunity.

12. Office Assessments…Post Assessment • Complete AS9104/2 Form L (CB Office Assessment Check Sheet) and provide copy to AB Team Leader ASAP! • When requested by the AB, please provide timely feedback/guidance(e.g. Corrective actions, appeals) • Never retain copies of CB records. • Don’t share confidential information back at your OEM! • The hat you are wearing is the AAQG/IAQG, Not your OEM!

13. CB Witness Assessments “Joint” Assessments with ANAB (AB)

14. Witness Audits…Planning • Acknowledge/confirm assessment assignment from ANAB coordinator (ANAB will then have confidence on assignment) • Communicate with AB Team Leader (Travel logistics, etc...) • If you have not heard from the AB Team Leader, reach out to them, an email may have been missed/lost!

15. Witness Audits…Planning • Communicate with ANAB coordinator • Will send you pre-audit information once available (e.g. CB Plan, CB Client Profile Form) • Review CB Client Profile form to familiarize yourself with the organization that is being audited • Personal Safety (e.g. Safety Shoes, Glasses) • Security requirements (e.g. ITAR, EAR)

16. Witness Audits…Planning • Again, don't make assumptions (take ownership of all travel details) • Don't come into the assessment with no knowledge of any safety or security requirements • If you did not pick up on security clearance items you may be asked to leave and not be allowed in the building! • Don’t let that happen to you!!!

17. Witness Audits…Execution • Audit begins with AB’s opening meeting with the CB • At this point you can only “talk” to the AB team…you can not ask questions or discuss the audit, standard, etc with the CB team or CB’s client. • Observe and jot down any questions you may have so that they can be addressed as appropriate after the AB Team Leader has completed their opening meeting

18. Witness Audits…Execution • Opening meeting between CB Auditor and CB client • A brief amount of time is typically allocated to the AB Team Leader to make introductions and describe AB and OP objectives. When asked, feel free to briefly summarize your background, company, etc. • Remain in the background, observe, and take detailed notes • Turn off or silence all electronic equipment

19. Witness Audits…Execution • Again…Remain in the background and remain quiet, don’t talk! • Observe and take detailed notes, they will be needed! • Don’t touch anything (e.g. parts, dwgs, paperwork) • Stay with the AB Assessor and CB auditor at all times • Don’t encroach on the CB auditor and/or auditee

20. Witness Audits…Execution • Avoidconversations / eye contact • Do not express any body language to signal good or bad (rolling of the eyes, heavy sigh’s, smiling, etc…) • Avoid phone conversations/messaging (step away if the call is important) • Hold all thoughts and concerns until they can be discussed between yourself and the AB Assessor and/or AB Team Leader in private

21. Witness Audits…Execution • Have your notes ready to discuss with AB team to document findings, etc • Support the AB team during the AB/CB closing meeting when needed • Don’t wander off • Don’t discuss the standard and requirements, even at a break/lunch. • Recognize what actions/words influence the audit!

22. Witness Audits…Execution • Do NOT actively audit! • Doesn’t matter what you see/hear/feel; do NOTactively audit! • Do NOT influence the outcome of the audit in any way, shape or form! • Do NOT fall asleep! • Do NOT start interviewing personnel or inspecting parts! • Get the idea?

23. Witness Audits…Execution • Make sure your notes are clear and concise to discuss with the AB team, there is not a lot of caucus time for the AB/OP Assessor • Do not argue with the AB team…any disagreements bring back to the AB and RMC Chair.

24. Witness Audits…Post Assessment • Complete the AS9104/2 Form M (CB Witness Assessment Check Sheet) ASAP • When requested by the AB provide feedback/guidance (e.g. Corrective actions, appeal) • Do not share information found during the witnessed audit! • All information is confidential!

25. Witness Audits…Post Assessment • Do not follow-up with the organization for a job! • Do not go back to your procurement department about information on this organization.

26. Overall • The majority OP assessors • Understand the process, • follow the rules, • understand the requirements and • represent the AAQG and your industry very well • You should be proud!!!

27. Overall • Don’t go out if you are not ready, willing, able AND prepared to follow the process, requirements and do a good job! • Talk to your OEM if you have concerns! • A significant portion of this “guidance” came from real life experience….sadly!

28. ?? Questions ??

29. Presenter’s picture • Bob Cruse • ANSI-ASQ National Accreditation Board (ANAB) • 4 years US Navy (Grumman EA6B/Whidbey Island NAS) • 25 years The Boeing Company • 4.5 years 3rd Party Auditor (AQMS & ISO) • 45 years Rock N Roll drummer

30. GO Hawks!