Modernizing Accountability

1. Modernizing Accountability A need for evaluation Presentation to the CES 2003 Annual Conference Vancouver John Mayne john.mayne@oag-bvg.gc.ca June 4, 2003

2. Outline • Modernizing accountability • The accountability process • Keys to effective accountability • The many roles for evaluation Office of the Auditor General

3. Traditional accountability • Accountability is not a simple concept • Its meaning & how it works often disputed • Traditionally tied to hierarchy • Traditionally, there has been accountability for: • Resource inputs, activities and outputs • Adherence to rules and procedures • Actions taken – or not: “blame culture” Office of the Auditor General

4. Modernizing accountability • Pressures for change in public sector management and governance • Focus on results – distinguishing outcomes from outputs • Increased use of partnering arrangements • Increased discretion, flexibility and innovation (risk-taking) Office of the Auditor General

5. Purposes of accountability • Control against the abuse of power • Provide assurance that activities are carried out as intended, with due regard for fairness, propriety and good stewardship • Encourage improved performance by reporting on & learning from what works – or does not Office of the Auditor General

6. Accountability revisited Accountability is a relationship based on obligations to demonstrate, review, and take responsibility for performance, both the results achieved in light of agreed expectations and the means used. Office of the Auditor General

7. Principles of effective accountability • Clear roles and responsibilities • Parties understand and agree upon them • Clear performance expectations • Objectives, expected results & constraints • A balance of expectations with capacities • Linked to each party’s capacity to deliver • Credible reporting • Demonstrating achievements & learning • Balanced: both good news and bad – not just good • Reasonable review of performance • with adjustments, corrections & consequences for individuals (rewards & sanctions) Office of the Auditor General

8. The accountability process Office of the Auditor General

9. Accountability framework • Clear roles and responsibilities • Clear and realistic expected performance • Credible reporting requirements • Effective mechanisms for review & adjustment In the context of transparency & public service values Office of the Auditor General

10. Holding to account How are those responsible being held to account? • Reporting− credible, understandable and timely • Financial and non-financial results • Means used to achieve those results (fairness, propriety & good stewardship) • Both good news and bad • Review & adjustment • Review & analysis of performance • Necessary adjustments: Something has to happen Office of the Auditor General

11. Keys to effective accountability • Fostering a climate for valuing results • Designing practical yet challenging accountability frameworks • Reliable measurement of results, and means • Learning not to blame; results-based learning • Setting outcome performance expectations • Reasonable accountability for outcomes • Implementing accountability practices Office of the Auditor General

12. Setting expectations • essential for accountability • traditionally involves specifying a few clear and concrete output/outcome performance levels • would be better to • present and discuss the results chain • agree on some clear and concrete outputs and outcomes within the results chain to be accomplished • performance expectations results chains Office of the Auditor General

13. Accountability for outcomes Need to credibly demonstrate: • The extent to which the expected results were achieved • The contribution made by activities and outputs of the program to the outcomes • The learning and change that have resulted, and • The soundness and propriety of the means used. Office of the Auditor General

14. Outcomes and attribution • Outcome results-focus requires credible assessment of program’s contribution • Contribution analysis - not an exact science • Logic and empirical evidence can provide a reasonable basis for attribution • Looking for plausible association Office of the Auditor General

15. Evaluating accountability • Is there an appropriate framework? • Roles and responsibilities • Expected performance • Reporting requirements • Mechanisms for review and adjustment • Is the arrangement being held effectively to account? • Credible reporting • Closing the loop (reviewing and adjusting) Office of the Auditor General

16. Conclusion • Move beyond blame • Hold those responsible for public policy effectively to account • For the results – and improvement through learning • For the means used to achieve the results • All based on sensible measurement (evaluation and monitoring) of results and contribution Office of the Auditor General

17. OAG references Modernizing Accountability in the Public Sector. Report of the Auditor General of Canada to the House of Commons, Dec 2002, Chapter 9. http://www.oag-bvg.gc.ca/domino/reports.nsf/html/20021209ce.html/$file/20021209ce.pdf Addressing Attribution Through Contribution Analysis: Using Performance Measures Sensibly. OAG Discussion Paper, 1999 http://www.oag-bvg.gc.ca/domino/other.nsf/html/99dp1_e.html Also in Canadian Journal of Program Evaluation, Vol. 16, No. 1, 1-24. Reporting Performance to Parliament: Progress Too Slow. Report of the Auditor General of Canada to the House of Commons, 2000, Chapter 19. http://www.oag-bvg.gc.ca/domino/reports.nsf/html/0019ce.html/$file/0019ce.pdf Reporting on Outcomes: Setting Performance Expectations and Telling performance Stories. OAG Discussion Paper, 2003.http://www.oag-bvg.gc.ca/domino/other.nsf/html/200305dp1_e.html/$file/200305dp1_e.pdf Office of the Auditor General