Results-based Management

1. Results-based Management Strategic Planning and Performance Measurement A Short Course October 17, 2008 Leslie M. Fox Consultant to the RBM-KM Task Force

2. Training Purpose & Objectives PURPOSE • As per MTSIP Action Plan: Refine MTSIP … make it SMARTER: Results & Indicators • To provide an understanding of and basic tools for use in developing strategic plans (SP) and performance measurement plans (PMP) that are RBM compliant OBJECTIVES • First step is to develop the component pieces of MTSIP … Focus Area SPs and PMPs • This short course is designed to begin that process so that you / we can work iteratively and, at times, virtually • Over the next 2 to 3 months FA SPs and PMPs will be prepared which in turn form the basis of revised MTSIP

3. Training Programme Content • Review of RBM at UN-HABITAT • Initial findings, conclusions & recommendations • RBM Principles and UN-H conformity • Strategic Planning: Process & Tools • The how and why of strategic planning • SP best practice: The results framework • An illustrative MTSIP results framework • Types of Results: Not all results are equal • Developing a Focus Area Strategic Plan • The Structure and Contents of FA Strategic Plan • Some Practical Work on developing an FA results framework • Strategic Planning Tips • Performance Measurement Plans • Relationship to strategic planning • How to develop performance indicators and evidence-based methodology

4. Results-based Management: Initial Findings at UN-H • OIOS and my own analysis identified: • Broad mandate versus limited resources • too great of a focus on outputs (inputs / money and activities) • Too little focus on results, impact and performance … energy dissipated with concrete few results / long-term impact • Outputs and activities are about the: short-term • Achieving results and performance: long-term • Current planning and management structure, including financial and progress reporting, exacerbate the problem by focusing on biannual change. • It is virtually impossible to measure impact change over the short-term: 1 or even 2 year intervals • Strategies are measured over a period of three to seven years with impact an expected high level result

5. Initial Conclusions & Recommendations • Getting the fundamentals right • Refining and finalizing the results framework: • causal logic from mandate to outcomes to activities • Focus area strategic plans should be developed • Formulating SMART results & SMART indicators • Organizational structure for delivering results • Results achievement should be through the Focus Areas • IN AN IDEAL WORLD there would be no Divisions, just FA results Teams. • IN THE REAL WORLD all Divisions must be aligned with all FAs • Corporate level strategic planning & performance measurement unit • Using existing resources effectively • Make projects work for results, not the reverse • Aligning Resources (staff and financing) with expected results • Incentives should reward results’ achievement • Emphasize FA Results team accountability and empowerment • Incentives promote joint results achievement / info sharing • Change should be based on the Art of the Possible

6. RBM Principles & UN-H Conformity • Being Flexible versus Being Strategic • If It Is Not Measured Then It Did Not Happen • Answering the “So What” Question • RBM: Empowering Performers While Holding them Accountable • A RBM-oriented strategic plan and corporate marketing strategy • RBM embodies best practice: the MDG paradigm • Strategic Planning & Performance Measurement are Core Principles of RBM • A Results Framework (Best Practice) Is a Graphic Presentation of the MTSIP • Single Most Important Factor in Successful RBM Approach is Leadership

7. Strategic Planning: Why & How The Why of Strategic Planning • To bring order out of chaos / distinguish what is achievable from what is desirable • To move beyond “so what” outputs to the achievement of results and impact • To not only achieve results but measure them through an evidence-based methodology The How of Strategic Planning • Performance indicators measure results. • Results are determined in the process of developing a results framework (RF). • Developing a RF and formulating corresponding indicators to measure those results is undertaken during the process of strategic planning.

8. Principles of Strategic Planning • Strategic planning and performance measurement are 2 inter-related elements of RBM good practice • Articulate results that you agree to be held accountable for • And formulate corresponding performance indicators that measure progress towards the achievement of the results • Strategic plans are thus like performance contracts between an organization and its members / partners: • With the inputs (resources) provided we agree to produce the agreed upon set of results that will be measured by this set of performance indicators • Accountability and Empowerment • Strategic planning took place at the level of the MTSIP • Results or Expected Outcomes (EAs) and indicators developed • But the results of SP: results & indicators were not operationalized into a plan to measure progress towards achievement of these results and the performance of the responsible for their achievement • Performance measurement takes results and indicators – the outcome of SP – combines them with methodology to fashion a results’ monitoring plan

9. Results Frameworks: Strategic Planning Best Practice • Strategic planning is a process of logical sequences of results and outputs (and inputs) • It is based on developing a hierarchical set of causal relationships both vertically and horizontal that show how change takes place • Lower level results must be achieved for higher level results to be achieved • If this change or series of changes are achieved … then it will lead to a corresponding change(s) • Results Frameworks are the tool that is best able to help us define these causal relationships • Most UN-H SP has used LogFrame methodology. • It is linear and does not show causal relationships and thus help us to better understand the logic of change

10. UN-HABITAT MANDATE: HABITAT AGENDA / MDG 7 : TARGETS 10 & 11 / INT’L CONFERENCE PRIORITIES UN-H GOAL SU created by cities & regions that provide citizens with adequate services, security & employment opportunities MTSIP Vision To help create, by 2013, the conditions for international & national efforts to stabilize growth of slums and to set the stage for the reduction of urban poverty and the number of slum dwellers MTSIP Strategic Objective Sustainable Urbanization in targeted countries is a fundamental principle driving the making and implementation of development policy at the national and local levels in an engendered and youth sensitive way MTSIP EA / Outcomes Focus Area 1: Effective normative work informs SU policy making in … Focus Area 2: Participatory processes strengthen SU Partners to … Focus Area 3: Enabling policy / legal environment promotes PP land … Focus Area 4: Expanded access to environmentally sound urban services Focus Area 5: Increased access by the poor to innovative financing systems Focus Area 6: UN-H staff are empowered to achieve planned results MTSIP Sub-EAs / Outcomes Sub EA 1.1 Sub-EA 1.2 Sub-EA 2.1 Sub-EA 2.2 Sub-EA 3.1 Sub-EA 3.2 Sub-EA 4.1 Sub-EA 4.2 Sub-EA 5.1 Sub-EA 5.2 Sub-EA 6.1 Sub-EA 6.2 MTSIP Outputs MTSIP Activities / Projects • A • B • C • D • A • B • C • D • A • B • C • D • A • B • C • D • A • B • C • D • A • B • C • D MTSIP Inputs Regular Resources = 10 percent / Non-Earmarked Project = 10 percent / Earmarked Project = 80 percent UN-HABITAT Medium-Term Strategic & Institutional Plan (MTSIP) Results Framework

11. Types of Results: All Results are not Equal Strategic Objective / MTSIP Highest Result: • the highest level result that a program strategy believes it can materially affect through its direct actions or performance; • takes responsibility for achieving with the resources at its disposal; and, • agrees to be held accountable for attaining the results that it formulated. Outcomes or Expected Accomplishments: • lower or intermediate results required to achieve the program strategic objective. • There can be several levels of outcomes, including those that correspond to Focus Area strategic objectives Outputs – Activities – Inputs: • Outputs are NOT results! They are the outcome of inputs and activities … So What! • These are largely what are measured in your biennial work programmes and AWPs • RBM performance measurement systems do not care about outputs • They are a management tool at best Programme Goal – Impacts: • Goals are not results and are not measured in a RBM PMP. • Rather, goals are aspirational, a desired end state but beyond the immediate reach of a relevant or concerned strategic planning period. • A given results framework contributes, along-side, many other initiatives and most likely over several strategic planning periods to the achievement of a program goal.

12. Developing a Strategic Plan • There are many formats – structure and content – for the development of a SP • Step 1: FA situational analysis: SWOT, mapping external environment, results achievement, previous experience, lessons learned, etc. • Step 2: Formulating the FA Strategy: Results Framework including results / indicators, outputs (programmes and projects) and required inputs – Risks & Assumptions • Linking FA strategy to the HABITAT Agenda, MDGs, etc. • Linking FA strategy to MTSIP Goal and strategic objectives • Linking country programmes to FA strategy • Step 3: FA Management Plan, implementing partners, contractual arrangements, etc. • Step 4: Performance measurement plan • Indicators plus methodology: data collection, analysis & reporting

13. Focus Area Strategic Planning:Structure and Contents • COVER PAGE • SUMMARY • SECTION: BASIC DATA TABLE: (Relevant data / information) FA Indicators that portray status of change variables (e.g., urban poverty rates, slum growth, etc). • Effective advocacy, monitoring and partnerships • Promotion of participatory urban planning, management and governance • Promote pro-poor land and housing • Environmentally sound basic urban infrastructure and service • Strengthened human settlements finance systems • FA Indicators (Relevant data / information)

14. Focus Area Strategic Planning:Structure and Contents • SECTION: FOCUS AREA SITUATION • SECTION: KEY RESULTS AND LESSONS LEARNED FROM PREVIOUS PROGRAMMES 2006- 2008 • Focus Area Strategic Result • An "Expected Accomplishment" or “outcome-level” result • Sub-section: Lessons learned from Past Cooperation • SECTION: THE FA STRATEGY, 2008 – 2013Sub-section: Summary Budget Table • The Sections are: Regular Budget, General Purpose (Foundation), Specific Purpose (Foundation), and Technical Cooperation aggregated for the period. Focus Area Regular Budget GP (F) SP (F) TC Total • Component 1 • Component 2 • Component 3 • Component 4 • Monitoring and Evaluation • Total

15. Focus Area Strategic Planning:Structure and Contents • SECTION: THE FA STRATEGY, 2008 – 2013 (continued) • Sub-section: Preparation Process • Sub-section: Goals, Key Results and Strategies • Sub-section: Relation to HABITAT Agenda • Sub-section: Relationship to International Priorities • Sub-sections: FA (Sub-Programme) Components • SECTION: Monitoring, evaluation and FA management:

16. Strategic Planning Tips SMART Definitions • Specific: the result is clearly stated and described in change language; • Measurable / Monitorable: an assessment is possible to decide whether the result has been achieved; • Achievable: the result correlates to a target that can feasibly be attained by the program partners with UNICEF support. All necessary resources are identified and budgeted for; • Relevant: the planned result represents a milestone in the results chain, leading to the achievement of commitments related to the Millennium Declaration, and national priorities; • Time-bound: the achievement of the results is likely to happen within the plan period. There is an expected date of accomplishment, usually by or before the end of the program cycle.

17. Strategic Planning Tips Planning Assumptions and Risks In order to test the vertical logic (soundness of the Results Framework) of programme strategies, it is necessary to refer back to the original analysis of the problem and the contributions of others. … There is a planning assumption, that the other partner will do what is necessary and do it successfully. Assumptions may fail. When critical planning assumptions fail, the attainment of the Strategic Result may be at risk. The Results Framework identifies such critical planning assumptions, so the planning teams can deal with them and reduce the risk of failure.

18. Strategic Planning Tips Lessons Learned • Possibly the most important criterion for making choices during program design is well-founded experience from the past. Strategies often continue to be replicated without their validity being confirmed through evaluation, or other methods such as participatory reviews. • Therefore, if a specific strategy or program is proposed for continuation in the a new FA (or strategic framework), it should have been formally evaluated. • At a minimum, the FA teams and national or local partners should take stock of what worked and what did not, and concentrate resources in the most promising program areas.

19. Developing an FA Results Framework • The first step is to articulate the MTSIP strategic objective – Help please! • Second step is formulating the FA strategic result – Help please! • Third step is to formulate the outcomes or expected accomplishments – several levels of outcomes are possible • Fourth step is to develop corresponding outputs and activities for each of the lowest level results/outcomes/EAs • For every Result/outcome EA formulated for the FA there should be at least two performance indicators

20. FOCUS AREA 1: Effective advocacy, monitoring and partnerships FA 1 Goal / MTSP SO Sustainable Urbanization in targeted countries is a fundamental principle driving the making and implementation of development policy from global to local levels in an engendered & youth sensitive way FA 1 Strategic Result Effective Normative Work Informs Sustainable Urbanization Policymaking and Implementation ExpectedAccomplishments/ Outcomes-level Results EA 1 EA 2 EA 3 Sub-EAs / Sub-Outcomes Results Sub-EA 1.2 Sub-EA 2.1 Sub-EA 3.1 Sub-EA 3.2 Sub-EA 1.1 Sub-EA 2.2 FA / EA Outputs output output output output output output output output output output output output FA Activities • Illustrative Activities • Illustrative Activities • Illustrative Activities

21. Performance Measurement Planning: Results-based Management • Performance measurement is an integral part of and directly linked to strategic planning • SP: formulation of results & performance indicators • PMP: methodology for collecting, analyzing and reporting on data related performance indicators which measure results • UN-H + GB + international partners are committed to RBM approach (MTSIP / Action Plan) – Aid effectiveness • Is concerned with results not inputs & outputs • Measuring and reporting is evidence-based • A good start on MTSIP: additional refinements on results and indicators is necessary • Next step for FA teams is to develop PM Plans • This will require finalizing results and indicators and developing methodology for each performance indicator articulated • We have begun discussions and will finalize over next 2 months

22. Performance Measurement Plans • First step in formulating results and indicators is to develop a results framework (see above) • Next 2 months will spent finalizing FA SPs / results framework to begin formulating performance indicators. • Once we have determined each FA’s results and corresponding performance indicators, then we have the core elements to develop performance monitoring and evaluation plan (PMP). • In addition to these two core elements of the PMP, the methodology of how to collect, analyze and report on data, or the who (collects), when (how often), where, and how (survey’s, secondary data) of data collection, analysis and reporting is required • The purpose of evaluations is different in an RBM context: they are undertaken independently of performance measurement and focus on a range of subjects (e.g., continuing validity of results framework)

23. PMPs Continued • Each performance indicator must have an initial baseline data figure, or the point in time before, preferably, the initiative or project commences that is designed to achieve a given result. • Unless we have the baseline it will be impossible to measure change that results from the interventions / activities that are undertaken to achieve the concerned result. • Secondly, a performance indicator must have a target, or the expected degree of change of a given result that is to take place over the life of a strategy, a program or a project. Targets are normally expressed as either a percentage or a number.

24. PMPs Continued • In developing a performance indicator we use a performance management data sheet (see below) which allows us to include all the above information, including the corresponding result, baseline data, targets, description of indicator and the methodology to be used. • Each result should have no more than two (up to 5 for FA strategic results) performance indicators. • Where it is possible, a qualitative and quantitative indicator should be paired to measure result achievement.

25. PMPs (Continued) • The development of indicators is an iterative process in which one develops the indicator and then sees if there is data that can be collected at a reasonable cost to measure change. • Indicators should be disaggregated by gender, global/region/national, youth, minorities or other characteristics related to the nature of the result or change being sought.

26. Performance Measurement Tips The Horizontal Logic • Examines program performance assessment elements: • Avoid combining several results into one statement. Specify statements about the quality, improvement, or implementation of a policy, legislation, or service in verifiable terms. • If no measurable indicator can be found that describes the planned results, the result will have to be reformulated (e.g. “strengthened capacity” would need to be specified). • Without a baseline, it will not be possible to detect change resulting from the interventions. (e.g. “70 % of children eat iodized salt” is not a meaningful result if the current status is not known – it might already be at 75 %!). • Well-formulated results contain targets (e.g. “improve coverage” is not an adequate statement to assess whether the result has been achieved). • Without Means of Verification (MOV), the achievement of the result cannot be assessed. MOVs should be reliable, practical and cost-efficient. This column will indicate whether the result can be verified through reviewing existing information systems, or whether a survey, study, special review process … is required.