The Peer Review Process

1. The Peer Review Process Presented by: Alan Ash, City Auditor Gainesville, FL Region Coordinator for N.A.L.G.A. Peer Reviews Florida West Coast Chapter of the IIA – Governmental Seminar February 3, 2004

2. Presentation Overview Overview of the Peer Review Program • Why have a Peer Review? • Variability/Flexibility Built into the Process • What Reviewers Review • What Happens During the Peer Review • Reporting Results • Preparing for Peer Review

3. Why Have a Peer Review?

4. Quality Control Standards GAS 3.49 “Each audit organization performing audits and/or attestation engagements in accordance with GAGAS should have an appropriate internal quality control system in place and should undergo an external peer review.

5. Quality Control Standards Frequency of Peer Review using Government Auditing Standards “ 3.52 Audit organizations performing audits and attestation engagements in accordance with GAGAS should have an external peer review of their auditing and attestation engagement practices at least once every 3 years by reviewers independent of the audit organization being reviewed. “

6. Other Reasons • Credibility • Public/Management Relations • Who’s Auditing the Auditors? • Improves Audit Quality • Provides for Consistency in Audit Work • Sets a Bar/Expectations • Professional Development

7. Reasons Not to Have Peer Review • Risk in receiving a poor report • Not a political necessity • Too Expensive • Polarizing Audit Staff • Do Not Follow Standards • No Time • Cannot meet Standards • Withstand Legal Scrutiny

8. Recent N.A.L.G.A. SurveyQuestion How many have participated in Peer review?

9. N.A.L.G.A. Survey 36% of respondents participated in a peer review program within the past 3 years. N = 150

10. No Peer Review Because: 19 lack of funds 17 do not follow YB 10 no time 8 cannot meet YB and would not risk adverse opinion

11. What does this Mean? Peer Review process allows for flexibility in how audit shops meet Yellow Book standards.

12. Variability in Internal Control Systems 3.51 "The nature and extent of an organization's internal quality control system depend on a number of factors, such as its size, the degree of operating autonomy allowed its personnel and its audit offices, the nature of its work, its organizational structure, and appropriate cost-benefit considerations. Thus, the systems established by individual organizations will vary as will the need for, and extent of, their documentation of the systems."

13. What Reviewers Review

14. WHAT’S REVIEWED “The peer review should include a review of the audit organization’s internal quality control policies and procedures, including related monitoring procedures, audit and attestation engagement reports, audit and attest documentation, and other necessary documents. (for example, independence documentation, CPE records, and personnel management files related to compliance with hiring, performance evaluation, and assignment policies). The review should also include interviews with various levels of the reviewed audit organization’s professional staff to assess their understanding of and compliance with relevant quality control policies and procedures.” --Government Auditing Standards, 2003 Revision, pages 3.54 (a)

15. What Happens During Peer Review……the big picture.

16. Objectives of Peer Review The peer review should be sufficiently comprehensive to provide a reasonable basis for concluding whether the reviewed audit organization’s system of quality control was complied with to provide the organization with reasonable assurance of conforming with professional standards in the conduct of its work. --Government Audit Standards, 2003 Revision, page 3.54(C)

17. Reviewer Qualifications • Reviewers should be qualified and have current knowledge of the type of work to be reviewed and the applicable auditing standards. • Reviewers should have a thorough knowledge of the government environment and government auditing relative to the work being performed.

18. Reviewer Quals (Continued) • Reviewers should be independent of the audit organization being reviewed, its staff, and its auditees whose audits are selected for review. An audit organization is not permitted to review the organization that conducted its most recent external quality review. • Reviewers should use sound professional judgment in conducting and reporting the results of the external quality control review.

19. Audits Selected for Review Reviewers will select audits that provide a reasonable cross-section of the audits conducted in accordance with GAS.

20. Reporting Results A written report will be prepared communicating the results of the external quality control review. Issued to Audit Management Audit Management Prepares Written Response.

21. Peer Review Workpaper Disposition In a N.A.L.G.A. peer review, reviewers are also required to maintain a record of their work in the form of working papers, and to include the audit organization’s written response in their issued report.

22. Optional Meetings with Elected, Appointed Officials and Management At the discretion of the office being reviewed, the review team may meet with executives of the organization to obtain their assessment of the office’s performance, promote the benefits of peer review, and point out that the office is achieving accountability by having a peer review.

23. Suggestions for Preparing for a N.A.L.G.A. Peer Review • Incorporate the N.A.L.G.A. Review Guide into your office manual and Audit Engagement Checklist into audit workpapers. • Use ideas, including forms and audit procedures, from other audit shops that have passed a N.A.L.G.A. peer review. • Volunteer auditors to serve on peer review teams.

24. Suggestions for Preparing for a N.A.L.G.A. Peer Review • Conduct a self-assessment using N.A.L.G.A.’s “Audit Organization Description of its Quality Control System”. • Speak with an experienced team leader to give you an informal opinion and suggestions regarding the adequacy of your quality control system.

25. Suggestions for Preparing for a N.A.L.G.A. Peer Review • Ask knowledgeable staff to oversee quality control: • Develop and update audit procedures. • Prepare for peer review by reviewing audit files. • Stay involved in the coordination of the external quality control review.

26. Peer Review Costs Audit Organization Pays • Travel • Lodging • Meals Other Considerations • Organization agrees to participate in Peer Review • No Fee paid to Reviewers • Audit Organization does not pay salaries of team members

27. QUESTIONS?