Building Quality Assurance Into Your Audit Activity

1. Building Quality Assurance Into Your Audit Activity Audit Process Versus Program The Difference…and why it Matters Brian Kruk | CIA, CISA, CGAP, CCSA, CCA Senior Director Quality Assessment Services

2. Agenda • Discuss the available QA&IP guidance • Examine common misconceptions in QA&IP development • Explore the differences between basic internal audit processes and effective components of a QA&IP • Utilization of the Old IIA PA 1311-2 to create an appropriate, right-sized QA&IP • Understand how a CMM can be used to facilitate the path to quality

3. “ Quality is not an act – it is a habit. - Aristotle ”

4. Quality Assessment • The process of evaluating the efficiency and effectiveness of an internal auditing organization through a comprehensive, qualitative review of audit procedures,leading to recommendations for improving controls, reducing risk and the introductions of successful innovative best practices. • It should also ensure compliance with the International Standardsfor the Professional Practice of Internal Auditing and other relevant organizational and departmental policies and procedures.

5. Why is QA&IP Important? • Reasons for setting up QA&IP • Know where your group stands at all times • Potential external QA cost savings • Reduce risk of external QA “surprises” • Improve the IA environment/process • Reasonable assurance to audit committee • Quality does matter (i.e. Org. initiatives and SOX) • Required by the Standards What reasons do you see out there?

6. What can QA&IP Tell You? • How things can be improved? • How more can be accomplished? • If maximum value is being received for your organization's investment in IA? • If you meet professional standards? • How you can add more value to the organization? • How you can enhance IA’s image, perception and credibility within the organization?

7. QA Related Standards 1300 – Quality Assurance and Improvement Program (New) The chief audit executive must develop and maintain a quality assurance and improvement program that covers all aspects of the IAAand continuously monitors its effectiveness. This program includes periodic internal and external quality assessments and on-going monitoring. Each part of the program should be designed to help the IAA add value and improve the organization’s operations and to provide assurance that the IAA is in conformity with the Standards and the Code of Ethics. Interpretation: • A quality assurance and improvement program is designed to enable an evaluation of the internal audit activity’s conformance with the Definition of Internal Auditing and the Standards and an evaluation of whether internal auditors apply the Code of Ethics. The program also assesses the efficiency and effectiveness of the internal audit activity and identifies opportunities for improvement.

8. QA Related Standards 1310: Quality Program Assessments • The internal audit activity should adopt a process to monitor and assess the overall effectiveness of the quality program. The process should include both internal and external assessments • New 1310 – Requirements of the Quality Assurance and Improvement Program The QA&IP must include both internal and external assessments.

9. QA Related Standards Old 1311 – Internal Assessments • Should include: • Ongoing reviews of the performance of the IAA. • Periodic reviews performed through self-assessment or by other persons within the organization, with knowledge of internal auditing practices and the Standards. New 1311 – Internal Assessments • Must include: • Ongoing monitoring of the performance of the IAA. • Periodic reviews performed through self-assessment or by other persons within the organization with sufficient knowledge of internal auditing practices.

10. QA Related Standards (New IPPF) 1311 – Internal Assessments • Interpretation: Ongoingmonitoring is an integral part of the day-to-day supervision, review and measurement of the IAA. Ongoing monitoringincorporated into the routine policies and practices used to manage the IAA and uses processes, tools and information considered necessary to evaluate conformance with the DIA, COE and Standards. • Periodic reviews are assessments conducted to evaluate conformance with the DIA, COE and Standards. • Sufficient knowledge of IA practices requires at least an understanding of all elements of the IPPF.

11. InternalAssessment • OngoingReviews • Work PaperReviews • PerformanceEvaluations • Actual vs. BudgetedAnalysis • Various MonitoringMetrics • CustomerSurveys • PeriodicReviews • Self-Assessment • Annually – Covering all Standards over 5years • Quarterly/Semi-Annual – Portions of Standards eachyear • Assess compliance with IA ActivityCharter

12. InternalAssessment • Ongoing Assessments caninclude: • …routine policies and practices used to manage the IAactivity… • Engagementsupervision • Checklists and othermeans • Feedback from IAclients/stakeholders • Project budgets, timekeeping systems, audit plan completion, cost recoveries and other performance metrics (e.g. cycle times and recommendationsaccepted) • Conclusions, follow-up, andimplementation

13. InternalAssessment • Periodic Assessments (Snapshot InTime) • Non-routine special purpose reviews andtesting • More in-depth interviews & surveys of stakeholdergroups • May be performed via self-assessment or by other competent audit professionals withinorganization • May include self-assessments, preparation of materials and benchmarking subsequently reviewed byothers • Can facilitate & reduce external assessmentcosts • Conclusions, follow-up, andimplementation • CommunicatingResults • Report results to various appropriatestakeholders

14. QA Related Implementation Guides • IG1300 - Quality Assurance and Improvement Program • IG1310 - Requirement of the Quality Assurance and Improvement Program • IG1311 - Internal Assessments • IG1312 - External Assessments • IG1320 – Reporting on the Quality Assurance and Improvement Program • IG1321 - Use of “Conforms with the International Standards for the Professional Practice of Internal Auditing” • IG1322 – Disclosure of Nonconformance • PG – Measuring Internal Audit Effectiveness and Efficiency • PG – Quality Assurance & Improvement Program • Old PA 2120-2 Managing the Risk of the Internal Audit Activity

15. Structure of Implementation Guides • Getting Started • Considerations for Implementation • Specific Related Topics • Example: IG1311 – Internal Assessments • On-going Monitoring • Periodic Self-Assessment • Considerations for Demonstrating Conformance

16. QA&IP Design Individual exercise List 3 components or tasks performed by your IAA that you feel illustrate your working QA&IP.

17. QA&IP Design Program vs. Process Differing Perspectives

18. QA&IP Design IIA Sample QA&IP

19. QA&IP Design Deleted – Practice Advisory 1311-2 Establishing the performance measure process • The CAE should: • Identify critical performance categories • Identify performance category strategies and measurement • Establish process for measurements to be monitored, analyzed and reported • Ensure measures used are appropriate to size and type of IAA

20. QA&IP Design Identify critical performance categories • Suggested categories • Key stakeholder satisfaction • Internal audit processes • Innovation • Capability

21. QA&IP Design Key stakeholder satisfaction Who are the stakeholders? • Internal • Audit committee • Executive management • Operating management • Internal audit clients • Audit staff • External • External government bodies and/or regulators • External auditors

22. QA&IP Design Key stakeholder satisfaction • How do you identify stakeholders? Consider the following: • Products and services being provided • Extent to which organization is regulated • Relationship with internal and external parties • Nature of the organization (public vs. private)

23. QA&IP Design Key stakeholder satisfaction • Satisfaction levels must be assessed and gaps identified! • Interviews • Facilitated sessions • Questionnaire • Develop appropriate plan for corrective action • Execute, monitor and re-evaluate periodically

24. QA&IP Design Internal audit processes • Risk assessment • Annual and long-range planning • Engagement planning and preparation • Proper scope, objectives, timing and resources • Conduct using established methodologies and practices • On-going communications • Reporting • Follow-up • Consulting • Fraud investigations

25. QA&IP Design Innovation and capability • Training and competence • Documented training plan by position • Minimum annual training hours • Certification requirements and levels attained • Utilization of technology • Staff training goals • Audit staff satisfaction • Data extraction and analysis, automated work papers • Industry knowledge • Periodic staff interaction • Employee loan programs • Formalized rotation programs

26. IPPF – Practice Guide Measuring Internal Audit Effectiveness and Efficiency • Defining internal audit effectiveness and efficiency • Internal and external stakeholders • Internal audit performance metrics/measures of effectiveness and efficiency • Monitoring and reporting results

27. IPPF – Practice Guide Measuring Internal Audit Effectiveness and Efficiency • Selected Narrative – Executive Summary • “To maintain and enhance IA credibility, its effectiveness and efficiency must be monitored.” • “Identify key performance measures for IA activities that stakeholders believe add value and improve the organization’s operations.” • “Effectiveness and efficiency measurements can be quantitative and qualitative.” • “Adequacy of engagement planning and supervision.”

28. IPPF – Practice Guide Measuring Internal Audit Effectiveness and Efficiency • Selected Narrative – Defining IA Effectiveness & Efficiency • “A general description of E&E is the degree (including quality) to which established objectives are achieved.” • “IA E&E should be monitored and assessed periodically as part of the IA process.” • Selected Narrative – Internal & External Stakeholders • “Specific feedback will provide insight into understanding of purpose, adequacy, deliverables, expectations, priorities, and shortcomings.”

29. IPPF – Practice Guide Measuring Internal Audit Effectiveness and Efficiency • Selected Narrative – IA Performance Metrics/Measures of E&E • “Audit activity can lead by example with strong, relevant, and reliable performance measures.” • “Identifying critical performance categories such as stakeholder satisfaction, IA processes, and innovation and capabilities.” • “Routinely monitoring, analyzing, and reporting performance measures.”

30. IPPF – Practice Guide Measuring Internal Audit Effectiveness and Efficiency • Selected Narrative – Monitoring and Reporting • “E&E should report to stakeholders periodically.” • “Consistent processes are needed for gathering, summarizing and analyzing measurement data. Responsibility for performing and validating measurement data should be similar to any other audit engagement.”

31. QA&IP Implementation Implementation should include • Measuring alignment with IIA Standards, key strategic objectives and applicable laws and regulations • Timely gathering, summarizing and analyzing data • Ensure measurements kept current and consideration for changing expectations, conditions, priorities and objectives • Effective, efficient on-going reporting to stakeholders • Annual reporting on IA effectiveness to AC • Appropriate internal resourcing • Documented methodology • Staff involvement and buy-in

32. QA&IP Should Reveal the IAA is… • Efficient and effective • Structured and staffed appropriately • Has an approach that is adequate and meet stakeholder expectations • Fully complying with the Standards • Utilizes sound testing techniques, methods and technology • Considers innovative practices and adopted them, when appropriate

33. Guiding Concepts • Design a program that fits your IAA • Utilize available internal resources • Treat as a project, start with a detailed plan • Promote total team involvement • Hold regularly scheduled update meetings • Educate all constituencies (IA staff, executive management, and the audit committee) on objectives and progress • Make the process as transparent, objective and participatory, as possible • Conceptualize on synergies with external QA

34. Capabilities Maturity Model Example

35. 2012 Quality Related Practice Guide • Quality Assurance & Improvement Program

36. QA Related Practice Advisories Old PA 2120- 2 Managing the Risk of the Internal Audit Activity • Managing the risk of not achieving IA objectives • IA must manage its own risk • Three categories: audit failure, false assurance, and reputation risks • Where were the internal auditors? • IA can implement the practices to mitigate its risk: • QA&IP • Periodic reviews of audit plan • Effective planning • Effective audit design • Effective management review and escalation • Proper resource allocation • Six through 14 – additional topics of further guidance

37. Remember! “ You manage what you measure. - Brian E. Kruk ”

38. Questions? Thank You! • Brian Kruk | CIA, CISA, CGAP, CCSA, CCA • Senior Director Quality Assessment Services • Brian.kruk@Honkamp.com | 888.556.0123