Performance Auditing

1. Performance Auditing Alta F�lscher CABRI/WBI PFM Training Seminar 18-20 June 2007

2. What is a Performance Audit Differs from regularity audit Different approach, more ad hoc, less rigid, different skills But same insofar an independent, objective assessment Audit of 3 Es Economy Efficiency Effectiveness All three or any combination of three Of wide range of issues Aim to lead to improvements Two dominant approaches Performance oriented Problem oriented

3. An example of a performance audit Does the Drug Abuse Resistance Education (DARE) Program Work? DARE is a drug abuse prevention program that aims to reduce drug use among school-age children. The performance audit of the program measured the extent to which it was achieving its goals. The audit used an experimental design, comparing juvenile arrest rates for youth in the program (the intervention group) with those who did not (the control group). The demographic profiles (ethnicity, income levels, age) were identical. The audit found that students who participated in the program were arrested more frequently than the control group, for both drug-related and non-drug-related offences. The design of this audit was heavily dependent on the existence of sufficient reliable data for determining student involvement in the program and identifying their arrest information in the local juvenile correctional system. These conditions are often difficult to meet, unless such comparisons have been planned from the start.

4. Why is it important? Based on need for public accountability Insight into activities of public agency on behalf of stakeholders Reliable independent information on performance strengthens legitimacy and trust Basis for future decisions Incentives for change Better government spending, better services, better public accountability and better public management Are things done in the right way; are the right things being done?

5. The 3 Es For same level of quality Economy Is about keeping the cost low Minimising the cost of resources used or required Are means chosen most economical use of resources? Have human, financial and material resources been used economically? Is management of programme in line with sound admin and management? Spending less Efficiency Relationship between outputs and the goods, services and resources used to produce them Can I produce more from given resources, can I use less resources for a level of outputs? In practice often difficult to distinguish from economy Spending well

6. 3 Es Effectiveness Relationship between intended and achieved results of public spending Are aims being met by the means employed, outputs produced and impacts? Are the impacts really result of policy rather than other circumstances? Spending wisely

7. 3Es Assessment always relative to audit criteria (standards against which actual performance measured) Important choice in audit design Standards used to determine whether programme meets or exceeds expectations Audit scope and design interprets economy, efficiency and effectiveness concretely Criteria should be relevant, reasonable, attainable eg Historical; Similar programmes or other locations; Legislation; Accepted standard; Professional opinion Assessment of effectiveness particularly tough Starts with programme design and clear, concrete goals Ideally measurement before and after + control group In practice difficult Less ambitious Assess plausibility of assumptions in programme design and achievement of goals as set down (goal achievement analysis) Have objectives been reached, target groups reached, what is level of performance?

8. Mandate of Performance Auditor Important underpinning for work Mandate and goals should be defined in legal framework Specific regulations Auditor must be free to select audit areas within mandate and refuse commissions Professional quality of work important: sufficient resources to acquire skills Differs across countries Coverage of mandate Scope of audits Approach

9. Input Output chain

10. What to audit? Determine selection criteria country by country Added value important Important problem / problem area Risk Substantial budgets, sudden increase in budgets Traditional risk areas New, urgent activities Complex management arrangements Performance information not updated

11. Standards for Performance Auditing Well thought out plan very important Pre-study Audit proposal Identify issue and audit objectives (what will be audited and why) Develop scope and design of audit What issues, what are hypotheses, what type of study? Is issue auditable and worth auditing? Understanding audited programme Defining audit criteria Methodological planning Quality assurance, timetable, audit team and resources Field standards Communicative and analytical process Sufficiency, validity, reliability, relevance, reasonableness of evidence Scepticism Record-keeping and oversight Accessible, comprehensive reporting

12. Performance auditing chain

13. Examples of audit findings

14. Performance auditing in Sub-Saharan African Performance auditing can help build foundations for performance system But Keep it simple at start Basics should be in place (rule of law, clearly defined institutions with understood roles and responsibilities, budget systems and accounting systems) Financial audit function in place first Organisational independence, legal mandate, sufficient funding, strong leadership, ability to recruit and retain skilled staff, stakeholder support and professional audit standards Credibility Support at highest levels of government