260 likes | 424 Views
AQMS Accreditation Programs ANAB Findings. Atlanta, GA July 22 , 2010 Steve Holladay ANAB, Accreditation Assessor. Auditor Workshop Atlanta, GA July 22-23, 2010. Objectives. Provide an overview of the NCR’s identified from the witness audits. Present in process approach.
E N D
AQMS Accreditation Programs ANAB Findings Atlanta, GA July 22, 2010 Steve HolladayANAB, Accreditation Assessor Auditor Workshop Atlanta, GA July 22-23, 2010
Objectives • Provide an overview of the NCR’s identified from the witness audits. • Present in process approach. • Discuss steps that could have been taken to prevent the NCR’s. • Rules of engagement.
Goal • Provide knowledge/information to the AS auditor pool with the goal to improve auditor/CB performance that will reduce repeat NCR’s which will, in turn, add value to the assessment process and the industry as a whole.
Overall • 120 NCR’s in 54 Witnessed Audits (WA) • 2.22 Average per WA • 50 NCR’s in approx 40 Office Assessments (OA) • 1.25 Average per OA
Pre-Audit Assignment • 3 NCR • DETAIL: • Assigning Auditors who either did not have the proper NACE code or AS91XX qualifications. • Client not made aware of the OP assessor. • LEARNINGS: • Verify auditors are qualified for the full dynamics of the audit. • Communicate clearly with the client.
Pre-Audit Planning • 11 NCR’s • DETAIL: 19011, 6.4.1.c • (4) The audit plan did not ID the organizations functional units and processes to be audited. • LEARNINGS: • The audit plan ensure the client will have the proper resources available to the audit. • Clearly demonstrates what processes are intended to be audited.
Pre-Audit Planning • 11 NCR’s • DETAIL: 17021. 9.3.2.1 • (2) Audit Planning not effective in assuring the AQMS is assessed to the min requirements for surveillance audits. • LEARNINGS: • Ensure the auditors understand where the full auditing requirements are for surv audits if not clearly ID in the audit plan.
Pre-Audit Planning • 11 NCR’s • DETAIL: MD 5 and AS9104 • (5) Insufficient Auditor Days in the planning without clear justification. • LEARNINGS: • If deviation…..JUSTIFY. • If the audit is being witnessed, provide to ANAB in the pre-audit information.
Pre-Audit Preparation • 5 NCR’s, several OFI’s. • DETAIL: • Information missing or not correct in EQM. • Client Profile form not complete or accurate. (scope, head count, regulatory, ITAR) • LEARNINGS: • Ensure the person entering information is knowledgeable about the information. • Verify with Client and lead auditor before submitting.
On-site Audit Depth of auditing • Total of 24 NCR’s • DETAIL: • Sampling part of an element and making a conclusion on the whole requirement. • LEARNINGS: • Audit plan and report accuracy is critical.
On-site Audit Depth of auditing • DETAIL: • Auditing to the scope of certification is not adequate. • LEARNINGS: • Ensuring the scope of certification, scope of audit and audit plan is consistent. • Validate the exclusions. Is scope consistent with the QM. • Ensure the address and scope on the Certificate is correct.
On-site Audit Depth of auditing • DETAIL: • Not following audit trails when objective evidence suggest otherwise. • LEARNINGS: • A plan is a plan. Follow the trail to its natural conclusion if potential findings are evident. • Keep head on a swivel. Don’t ignore clear findings.
On-site Audit Depth of auditing • DETAIL: • Not fully verifying the effectiveness of the actions taken on nonconformities identified during the previous audit. • LEARNINGS: • Ensure the evidence is more than a record review.
On-site Audit Interpretation • 4 NCR’s • DETAIL: • Design Applicability • LEARNINGS • More of an issue with AS9110.
On-site Audit Interpretation • DETAIL: • Not fully auditing Outsourcing when applicable. • LEARNINGS • Standard clearly requires “control of such outsourced processes shall be identified” and therefore subject to audit. Not just a document verification.
On-site Audit Interpretation • DETAIL: • Changes to the capability listing of the client. • LEARNINGS • Similar to the scoping discussion. Verify any changes to the capability listing as this could affect the scope of certification or introduce new processes and technology.
On-site Audit Decisions • 11 NCR’s • DETAIL: • Calling the Clients CA/PA system effective with ineffective/inadequate correction, root cause or corrective action. • LEARNINGS: • Deep dive the CA/PA system. Validate the information. If it doesn’t make since….make the call.
On-site Audit Decisions • DETAIL: • Soft Grading 8 NCR’s • Clearly stating findings during the audit and not raising the NCR or improper categorization. • Accepting informal correction during the audit. • LEARNINGS • Learn the definitions and follow them. Keep track of the “verbal” findings identified during the audit. Accepting correction to verbal findings without reporting is consulting. • As an auditor it is not your job to justify an NCR down.
On-site Audit Decisions • DETAIL: • Continuing the audit when the audit objectives are clearly unattainable. • LEARNINGS: • Only the client can make the decision to continue an audit. • There should be a clear conclusion that the objectives are unattainable and reported to the client with options. • At the time this is realized not at the end!!
On-Site Audit Closing • DETAIL: • Closing meeting does not address all of the requirements. • LEARNINGS • Clearly address the CA follow up and impacts on the existing cert. • Ensure the CA are presented to the correct requirements.
On-site/Off-site Audit Reporting • 22 NCR’s • DETAIL: • Accuracy of the Audit Report and Checklist to the observed audit. • LEARNINGS • Anecdotal evidence should be documented as such. Same for deductive evidence.
On Site/Off-Site Audit Reporting • DETAIL: • Report not in conformance with AS9014/AS91XX • LEARNINGS • Justify/explain all NA. • Clear evidence to support the conclusion. • Clear evidence to support the SCOPE. • Include detailed notes
Post Audit Reporting • 3 NCR’s • DETAIL: • Report did not contain mandatory items. • LEARNINGS • Ensure the Checklist are complete or references the location of the information. • Explain and differences between the information left on site to the published report.
Post Audit NCR Closure • 8 NCR’s • DETAIL: Improper closing of NCR’s and Improper Certification Decision. • LEARNINGS: • Clear evidence of Correction, RC and CA. • Ensure persons closing NCR are AEA. • Ensure the NCR’s are closed with appropriate evidence PRIOR to the cert decision.
Office Audit NCR’s • OASIS data base admin/accuracy • Verification of Client OASIS data admin • Justification of auditor days. • Improper qualification of AS cert decision maker. • Improper decision made with certification information.