Lessons Learned from Accreditation Assessments - ANAB Perspective

1. Lessons Learned from Accreditation Assessments - ANAB Perspective Boston, MA July 21, 2011 Steve HolladayANAB, Accreditation Assessor Auditor Workshop Boston, MA July 21-22, 2011

2. Objectives • Provide a summary of the NCR’s identified from the witness audits 2010. • Provide a summary of the NCR’s identified from the office audit in 2010. • Discuss four examples of the most common NCR’s. • Rules of engagement.

3. Goal • Provide knowledge/information to the AQMS auditor pool • with the goal to improve auditor/CB performance • that will reduce repeat NCR’s • which will, in turn, add value to the assessment process and the industry as a whole.

4. Overall • 2009 • 91 NCR’s in37 Witnessed Audits (WA) • 2.45 Average per WA • 19 Major NCR’s • 223 NCR’s in approx 37 Office Assessments (OA) • 6.03 Average per OA • 30 Major NCRs • 46 AS requirements • 2010 • 61 NCR’s in40 Witnessed Audits (WA) • 1.5 Average per WA • 22 Major NCR’s • 162 NCR’s in approx 42 Office Assessments (OA) • 3.85 Average per OA • 21 Major NCRs • 22 AS requirements

5. Witness Audit NCR’s • Audit team competency, Cert decision, notification of changes from client, OASIS admin, use of the mark. 1 NCR each. • Stage 1 objectives, Stage 2 objectives, Suspension process. 2 NCR’s each. • Certification Cycle, corrective action process, use of the Mark. 3 NCR’s each

6. Witness Audit NCR’s • Audit time and report issues. 4 NCR’s each. • Certificate errors and lack of verification. 5 NCR’s. • Collecting and verifying evidence. 6 NCR’s. • NCR recording and proper categorization. 11 NCR’s.

7. Office Audit NCR’s • Agreement to cover liabilities – 1 NCR. • Complaint Process – 2 NCR’s. • Certification agreement, certification cycle, satellite offices – 3 NCR’s each. • Surv-use of the mark evidence - 4 NCR’s • Application errors, monitoring of auditors, NCR recording/category, OASIS Admin -5 NCR’s each.

8. Office Audit NCR’s • Collecting/verifying evidence issues, financial stability evidence, records general – 6 NCR’s each. • Cert decision, Impartiality Committee representation, competency of auditors, multisite – 7 NCR’s each. • Audit report, NCR process – 9 NCR’s each. • Conclusion consistent with report – 12 NCR’s

9. Office Audit NCR’s • Transfers MD 2 issues – 13 NCR’s. • Audit time issues and justification – 15 NCR’s. • Option 2 conformance issues – 15 NCR’s • Certificate issues – 26 NCR’s

10. Case Study Examples • Four Examples of actual audit trails that have been sterilized. • The 4 examples represent approx 40% of the findings that occur during a witness audit. • Will present the scenario then present some choices the auditor could make. • Discuss the choices and select the BEST option. • Time management is KEY in this exercise. Its only Lunch after this session.

11. Example 1 • While auditing the purchasing process you find that there are external documents (ASTM) that are used in making decisions on what to specify for outsourced plating. The material comes back into the plant with a COA. The verbal answers to your question were all appropriate answers. • What are your next steps?

12. Example 1 • Verify conformance of control of external documents • Verify conformance of outsourcing • Verify conformance with process validation and product testing as it relates to the plated product. • all of the above. • None of the above. Anecdotal answers were acceptable. (best answer)

13. Example 2 • You observe a tank record of solutions analysis that has an incorrect date for change of tank. The record also had past notations of changes to the dates. Auditee says it’s a typo and changes it. You question why other information had appeared to be changed and was not consistent with what the procedure is required and are told that the company is reconsidering how they are using the form. • What are you next steps?

14. Example 2 • Increase the sample size. • Accept the explanation and continue the audit. • Raise a nonconformance. • Break early for lunch since it’s a HOT lunch today. (best answer)

15. Example 3 • The Audit Plan identify that FAI/Stamp control/configuration management/ Evaluation of Customer Contract Review are to be audited. As the audit proceeds you realize that there would not be enough time to complete the audit for these audit trails. • Your next step is?

16. Example 3 • Ask general questions without verifying all items on the plan in order to complete the plan. • Just ignore the rest of the plan since they were listed to help the customer understand where you might potentially go in the audit. • Document in the report or annotate the audit plan as a record what was not audited. • It’s a process audit. You can make inferences to conformance with what you saw in the doc control audit. (best answer)

17. Example 4 • During the course of the audit you record the following findings. • Internal Audits were not conducted on the status of importance when a process had repeat majors. • A Supplier CA was lost and never closed nor escalated for incorrect parts being shipped to the supplier. • Inaccurate root cause analysis was accepted without comment on 2 of 4 NCRs. What is your next step?

18. Example 4 • Note these points as OFI since the MR already knew about them. • Identify the findings as NCR’s. • Ignore them since you already have your limit of 10 NCR’s. • Discuss with the company what they should do to fix them so they will not be there for the next audit. (best answer)

19. Example Take Away • If findings are NCR’s….raise NCR’s. • Ensure the records are clear on what was audited and …..what was not. • Pursue Trails to their natural conclusion to determine conformance…and if the trails do not demonstrate conformance…its a nonconformance. • Ensure objective evidence is verifiable.

20. Questions