Workshop: Development and Use of Outcome-based Measures in Government Planning & Reporting

1. Workshop:Development and Use of Outcome-based Measures in Government Planning & Reporting September 28th, 2006 Winnipeg, Manitoba Presented by: Manitoba Treasury Board Secretariat , Office of the Provincial Comptroller of Manitoba, and The International Institute for Sustainable Development (IISD)

2. Agenda • The Context in Manitoba [10-10:30am] • Opening Statements and introductions • Overview of government-wide departmental planning and reporting • Trends in Planning, Reporting and Performance measures [10:30-11:15] • What are outcome-based measures and why are they important? • Innovative case examples • Sharing of Examples From the Department of Advanced Education and Literacy [11:15-noon] • Plenary discussions based on Pre-work assignment • Analytic Tools for the Development and Use Outcome-based Measures [noon-2:15pm] • Key steps in development, use and analysis of outcome-based measures • Group work on key steps Closing Remarks [2:15-2:30pm]

3. Objectives of Workshop • Intro to developing performance measures linked to organizational goals • Overview of current practice elsewhere • Understanding/discussion of issues related to government performance measurement reporting • Context for new Annual Report requirements (looking back) and new PSO requirements (looking ahead)

4. Part I. The Big Picture • Performance Measurement in Manitoba • Role of performance measures in annual reporting I. The Big Picture

5. Trends and context Trend toward reporting on changes in socio-economic and environmental conditions that matter to Manitobans I. The Big Picture

6. 2005 Discussion Document examples: Economy I. The Big Picture

7. 2005 Discussion Document examples: People I. The Big Picture

8. 2005 Discussion Document examples: Community I. The Big Picture

9. 2005 Discussion Document examples: Environment I. The Big Picture

10. The Planning and Reporting context • Performance Measurement is not done in isolation • Identified as part of Priorities & Strategies Overview (PSO) • reported upon in Annual reports • Performance Measures show whether our plans are working I. The Big Picture

11. Spring Yr1 Fall Yr1 Spring Yr2 Spring Yr3 Fall Yr3 PSO (plan) for following fiscal New initiatives for following fiscal Estimates for following fiscal Budget announced Estimatessupplement Execution of plans during fiscal year Annual Report Full cycle view I. The Big Picture

12. Spring Yr1 Fall Yr1 Spring Yr2 Spring Yr3 Fall Yr3 PSO (plan) for following fiscal New initiatives for following fiscal Estimates for following fiscal Budget announced Estimatessupplement Execution of plans during fiscal year Annual Report Full cycle view I. The Big Picture

13. Spring 04 Fall 04 Spring 05 Fall 05 Spring 06 Fall 06 Annual Report Budget Estimates supplement PSO (plan) Annual Report Budget New initiatives Estimatessupplement Estimates Budget PSO (plan) New initiatives Estimatessupplement Estimates PSO (plan) New initiatives Estimates Multiple cycles in play 04/05 05/06 06/07 07/08 I. The Big Picture

14. Issues/Opportunities re Annual Reports • Previous guidelines did not request measures of progress or performance • No specific requirement to link annual reports to larger process (Estimates Supplement or plans) • No specific requirement for Department annual reports to be placed online • New direction set in 2005 through Reporting to Manitobans on Performance I. The Big Picture

15. Trends andissuesin government annual reporting • Accountability • Who uses annual reports? (internet) • Public expectations (of results, of reporting) • Limited resources, demographic trends (retirements)… • …governments need to do “more with less” • Trend to shared, horizontal efforts • Important to agree on how to assess, report and use results

16. Spring Yr1 Fall Yr1 Spring Yr2 Spring Yr3 Fall Yr3 PSO (plan) for following fiscal New initiatives for following fiscal Estimates for following fiscal Budget announced Estimatessupplement Execution of plans during fiscal year Annual Report Recap: Full cycle view Annual Reports, including measures, are connected back to plans… I. The Big Picture

17. Part II. Trends in Government Planning and Reporting • What are outcome measures and why are they important • Concepts and vocabulary • Who is doing this really well? • Oregon: Results-oriented Strategic Planning • What are Alberta and Saskatchewan doing? • Alberta: Goal-based Budgeting • Sask. Government Accountability Framework II. Trends in Reporting

18. A. What are outcome measures and why are they important? Our Environment de•vel•op (di•vel′əp) v.t. 1. To expand or bring out the potentialities, capabilities, etc. Our People Our Economy II. Trends in Reporting

19. A. What are outcome measures and why are they important? Our Environment Experience has shown that a pathway to sustainability cannot be charted in advance. Rather, the pathway must be navigated through processes of learning and adaptation. National Academy of Science 1999. Our Common Journey: A Transition Toward Sustainability. de•vel•op (di•vel′əp) v.t. 1. To expand or bring out the potentialities, capabilities, etc. Our People Our Economy II. Trends in Reporting

20. IISD International and National-Level Perspective • Nations starting to develop information systems to gauge societal wellbeing and sustainability The United Kingdom Norway II. Trends in Reporting

21. Government Accountability Systems National SD Strategy Process Sustainable Development Principles Accountability Principles

22. Provinces and states using societal goal and outcome-based planning and budgeting systems Oregon Alberta IISD Provincial and State Level Perspective II. Trends in Reporting

23. Communities creating new forms of social infrastructure to navigate quality of life and sustainability Orlando • Healthy Community Indicators Initiativeof Greater Orlando Winnipeg • Quality of Life Indicators System concept being proposed by range of community stakeholders IISD Community-level Perspective II. Trends in Reporting

24. Logic Model Key Priority Areas Contributing to society’s wellbeing and sustainability objectives Change in wellbeing conditions (economic, social and environmental) High-level Outcome High-level Outcomes High-level Objectives Feedback Short to medium-term consequence of an output Intermediate Outcome External influences Output The result of an activity accountability Activity Processes and Inputs of a policy, program, or project Actions Hybrid from three different logic models (Canadian International Development Agency, Province of Alberta, State of Oregon)

25. Example Measures Social Environmental Healthy, sustainable surroundings Key Priority Areas Quality Jobs Children entering school ready to learn Change in well-being conditions (economic, social and environmental) High-level Objectives Stream water quality (turbidity) % children enrolled in Pre-K program Intermediate Outcome Short to medium-term consequence of an output Rate of soil erosion by water Demographic surveys for Head-start program completed Output Kilometers of river bank with vegetation The result of an activity Processes and Inputs of a policy, program or project Actions % of re-vegetation programs completed Head-start program implemented

26. Climbing the Steps toward Performance Management Analysis for Continuous Improvement Performance Measures Many agencies compare this month’s or this year’s performance measures to those of the past. Some are beginning to make comparisons with other agencies and to begin the process of benchmarking. Objectives Performance measures indicate how much or how well the agency is doing. Ideally, they track the agency’s progress toward achieving its objectives. Mission/Goals Objectives are unambiguous statements of the agency’s performance intentions, expressed in measurable terms, usually with an implied or explicit timeframe. Mission statements declare the agency’s long-range intent; its purpose. Although the goals expressed in a mission statement may help shape the agency’s values and its organizational culture, they often are imprecise and sometimes even a bit vague. (From Gov. of Alberta 1996) II. Trends in Reporting

27. Who does this really well?The Oregon Shines Case Study Oregon Shines Case Study

28. Oregon Shines Oregon's Strategic Plan- Oregon Shines (1989) - Updated every eight years - Encompasses the entire state Oregon Progress Board- independent agency created to be the steward of Oregon Shines - law mandates Board to report biennially - chaired by governor Oregon Shines Case Study From Conrad (2005)

29. Vision – “Oregon Shines II” Economy: Quality jobs for all Oregonians People: Safe, caring and engaged communities Environment: Healthy, sustainable surroundings Oregon Shines Case Study From Conrad (2005)

30. Oregon BenchmarksMeasures for how Oregon as a whole is doing. Key Priority Areas • Quality Jobs for All Oregonians • Economy (#1-17) • Education (#18-29) • Engaged, Safe & Caring Communities • Civic Engagement (#29-38) • Social Support (#39-60) • Public Safety (#60-67) • Healthy, Sustainable Surroundings • Community Development (#68-74) • Environment (#75-90) High-level Objectives Oregon Shines Case Study From Conrad (2005)

31. Linking Government to the Benchmarks Is society benefiting? High Level Outcomes (Benchmarks) Intermediate Outcomes Are strategies working? Is work happening? Outputs Oregon Shines Case Study From Conrad (2005)

32. http://www.oregon.gov/DAS/OPB/ Oregon Shines Case Study

33. Oregon Shines Case Study

34. Oregon Shines Case Study

35. Oregon Shines Case Study

36. Dept of Community Colleges and Workforce Development Oregon Shines Goals & Benchmarks Dept Goal #2: Oregon’s workforce is well trained and has access to a wide variety of training programs Goal #1: Quality Jobs for All Oregonians Key Priority Areas Percent of participants ranking WIA funded current workforce ratings good or better Benchmark #29 Change in well-being conditions (economic, social and environmental) High-level Objectives Intermediate Outcome Short to medium-term consequence of an output shared best practices, encouraged quality services and conducted quality assurance review Output The result of an activity Workforce skill development project Processes and Inputs of a policy, program or project Actions

37. External Influences Linking Government to the Benchmarks Oregon’s Progress Benchmarks Organization’s Progress Performance Measures Oregon Shines Case Study From Conrad (2005)

38. Towards Goal-based Budgeting in Alberta http://www.finance.gov.ab.ca/publications/measuring/measup06/index.html Alberta Case Study

39. Key Priority Areas for Alberta

40. Alberta Case Study

42. Department of Advanced Education Alberta Measures Up Dept Goal #1: High Quality Learning Opportunities for All Goal #2: Albertans will be prepared for Lifelong Learning and Work Key Priority Areas The learning system meets the needs of all learners, society, and the economy. Employment Rates of Albertans Age 24-35 by Highest Level of Education Change in well-being conditions (economic, social and environmental) High-level Objectives Intermediate Outcome Short to medium-term consequence of an output $3 billion Access to Future endowment; a $1 billion expansion to Alberta Heritage Scholarship Fund, $500 million expansion to Ingenuity Fund Output The result of an activity Introduced Bill 1, the Access to the Future Act, Processes and Inputs of a policy, program or project Actions

43. Part III. Sharing Examples • Sharing of Departmental examples in the use of outcome-based measures III. Department Examples

44. Manitoba Examples:Pre-Work Discussion • Review your pre-workshop assignment • Brief presentations [5 minutes + 5 min questions] • General discussion of issues III. Department Examples

45. Part IV: Development and Use of Outcome-based Measures • Overview of key steps in the development and use of outcome-based measures • Working through an example • Groups of at least two (per department if possible) IV. Outcome-based Measures

46. Logic Model Key Priority Areas Contributing to society’s wellbeing and sustainability objectives Change in wellbeing conditions (economic, social and environmental) High-level Outcome High-level Objectives Feedback Short to medium-term consequence of an output Intermediate Outcome External influences Output The result of an activity accountability Activity Processes and Inputs of a policy, program, or project Actions Hybrid from three different logic models (Canadian International Development Agency, Province of Alberta, State of Oregon)

47. Key Steps • Frame the logic model for your issue • Identify SMART measures for the logic model • Understand and articulate key external influences • Analyze feedback based on the SMART measures and external influences IV. Outcome-based Measures

48. 3. External Influences 1. Logic Model 2. Measures and Targets 4. Analysis & Feedback Key Priority Areas High-level Objective: _______________ Influences: Measure: Analysis: Analysis: Intermediate Outcome: _________________ Measure: Influences: Measure: Output: ________________ Analysis: Influences: Measure: Actions ________________ IV. Outcome-based Measures

49. 1. Logic Model Key Priority Area Safe drinking water, enjoyable recreation High-level Objective: Improved stream Water quality Intermediate Outcome: Reduced soil erosion Output: River bank re-vegetation Actions: River bank re-vegetation program Hybrid from three different logic models (Canadian International Development Agency, Province of Alberta, State of Oregon)

50. 2. Identify SMART Measures • Specific • Measurable • Aggressive, yet achievable targets • Relevant • Time-bound IV. Outcome-based Measures