Setting Key performance indicators to monitor progress

1. Setting Key performance indicators to monitor progress What are indicators Types of indicators Indicators at different levels Qualities of a good indicator Setting up indicators

2. Definition of an Indicator Specific information that provides evidence about the achievement of planned impacts, results and activities Ideally indicators should be reported quantitatively but this will not always be possible - don’t limit M&E to only what can be measured

3. What are indicators • Indicators are yardsticks that can be used to demonstrate that changes has or has not taken place. • They provide meaningful and comparable information to changes. • They are measurable or tangible signs that something has been done or that something has been achieved.

4. Examples of indicators • Indicator for improved standard of living-increased number of television aerials • Indicator for community empowerment-increased frequency of community members speaking at community meetings • Indicators are an essential part of a monitoring and evaluation system because that is what you measure.

5. Types of indicators • Indicators are either qualitative or quantitative criteria used to check whether planned changes have taken place as intended. • They (indicators) are designed to provide a standard against which to measure or assess or even show the success or progress of a programme against stated targets (GTZ&ITHOG 1989, Feuerstein 1986). Indicators have to be suggested, negotiated, adapted and approved.

6. Types of indicators • Quantitative indicators • Should be reported in terms of a specific number (number, mean, or median) or percentage. • Assessing the significance of an outcome requires data on both number and percent. • Qualitative indicators • Qualitative statements • Measure perceptions • Measure attitude, behavior

7. Quantitative indicators Examples • Number of • Proportion of • Percentage of • Amount of • The ratio of • Length of distance • Weight of • Size of • Areas of/spread of • Value of • etc.

8. Qualitative Indicators Examples • Level of • Presence of • Evidence of • Availability of • Quality of • Accessibility of • Existence of • Sustainability of • Improvement of • Ability to (e.g. skills) • Potential of • etc.

9. Types of indicators • The use of proxy (indirect) indicators • “better be approximately correct than precisely wrong”. • Proxy indicators are needed when it is difficult to measure the outcome indicator directly. • Used when data on the direct indicator is not available. • Used when data collection is expensive. • Used when it is not feasible to collect data at regular intervals.

10. Pre-designed indicators • The pros and cons of pre-designed indicators (Indicators that are established independently ). • MDGs, World Bank, IMF) • Pros • They can be aggregated across similar projects and policies • Reduce costs of building multiple unique measurement systems • Make possible greater harmonization

11. Pre-designed Indicators • Cons of predetermined indicators • They don’t address country specific or organizational goals • They are often viewed as imposed • They don’t promote key stakeholder participation • They can lead to the adoption of multiple competing indicators.

12. Indicators at different levels • Indicators are needed to monitor • Inputs, • Activities • Outputs • Outcomes • Goals

13. Outcome Indicators • Outcome indicators assess progress towards the project purpose and development goal: • Is the project making a difference • Target response • Benefits

14. Outcome indicators • Development Goal: Rural households on communal lands increase their average household income by 25% • Indicators • Increase in number of children from rural households enrolled in school • Decrease in incidences of malnourishment

15. Outcome indicators • Project purpose: • Subsistence farmers Increase their crop yields by implementing more appropriate crop production methods • Indicators • Farmers fertilize and fence their land. • Crop farmers plant and harvest a more diverse range of high grade vegetables. • Farmers sell meat milk and vegetables in the district markets

16. Output indicators • Is the project making any progress towards its objectives • How efficiently is the project making use of its resources. • Performance • Quantity, quality, timing, location, people • Efficiency • Were too many resources used

17. Indicators at Different Levels in the Objective Hierarchy • Impact indicators - indicators that show to what extent the project has contributed towards its goals • Result (Outcome and Output) indicators - indicators that show to what extent planned results (outputs and outcomes) have been achieved

18. Indicators at different levels • Activities (Process indicators) - indicators that show what activities have been completed • Input indicators - indicators that show what resources have been used by the project

19. Qualities of a good indicator • The “CREAM” of good performance indicators. • Clear- Precise and unambiguous • Relevant- Appropriate to the subject at hand • Economic-Available at a reasonable cost • Adequate-Provide a sufficient basis to assess performance • Monitor able-Amenable to independent validation

20. Qualities of a good indicator • Assessing proposed indicators • Is the indicator….. • As direct as possible a reflection of the outcome itself? • Sufficiently precise to ensure objective measurement? • Calling for the most practical, cost-effectivecollection of data ? • Sensitive to change in the outcome, but relatively unaffected by other changes? • Disaggregated as needed when reporting on the outcome?

21. Qualities of a good indicator In general, indicators should: • be verifiable (where and how to we get information about the indicator); • measure what is important and not what is easy to measure • measure only changes that can be linked or attributed to the project/programme; • be targeted in terms of quantity, quality and timing; • be useful in that valid and reliable (see p. …) data are available; • measure either quantitative or qualitative change.

22. How SMART? • As far as possible results should be: Specific Measurable Achievable Relevant (to the project purpose and goal) Timeframe • But don’t get too SMART… • What is achievable may need to be developed from experience • Good ideas take time to develop • Not everything that is worth doing can be easily measured

23. Ideal number of indicators • What is the ideal number of indicators? • The minimum number that answers the question • “Has the outcome been achieved” • “Has the output been achieved”

24. Developing Indicators Step 1: Identify the problem situation you are trying to address • Economic • Social

25. Developing Indicators Step 2: Develop a vision for how you would like the problem areas to look like • What will tell you that the vision has been achieved? • What signs will you see that will prove the vision has been achieved • This will give you impact indicators • If you can answer “yes” to these questions then progress is being made.

26. Developing Indicators Step 3 Develop a process vision for how you want things to be achieved • Participation • Enhancing community empowerment • This will give you process indicators

27. Developing Indicators Step 4 Develop indicators for effectiveness • Evidence compared to baseline • Indicators to show how you have effectively changed for example peoples attitudes and behaviors • This will give you outcome and output indicators

28. Developing Indicators Step 5: Develop indicators for efficiency targets • Indicators of time management • Resource utilization With this steps you are in a position to monitor and evaluate efficiency, impact ,…

29. Weaknesses in Indicator design • Indicators are usually discussed and defined late in the planning process (especially when this is confined to workshops). • Indicators are designed to meet scientific research needs and are therefore less appropriate and meaningful for the beneficiaries. • They are often not measurable in terms of cost-effectiveness. • There are usually too many indicators. • Indicators which are easy to measure are preferred. • Quantitative indicators are favoured to produce ‘hard and reliable’ statistics.

30. Weaknesses in Indicator design • They are usually not developed by stakeholders and/or beneficiaries and therefore do not represent their reality. • Indicators are donor driven since they have to legitimise the support for the respective project/programme. • Indicators are often an outcome of a desk study to prepare a proposal, satisfy funding requirements or to establish a baseline.