Poverty Reduction Information Systems (PRIS)

1. Poverty Reduction Information Systems (PRIS)

2. TOPICS FOR DISCUSSION • IMPORTANCE OF MONITORING & EVALUATION • DEFINITION OF TERMS: Goals, targets, indicators • TYPE OF INDICATORS • MONITORING • WHY AND HOW TO MONITOR • WHY EVALUATION • HOW AND WHAT TO EVALUATE • CAPACITY BUILDING IN M&E

3. Poverty Information systems are part of the Poverty Reduction Strategies • POVERTY MEASUREMENT & DIAGNOSTICS Understand the features of poverty and the factors that determine it. • POLICY ACTIONS Choose priority actions to reduce poverty, in the short- and longer-term. • INDICATORS AND MONITORING Set targets and identify indicators of progress, and systematically monitor and evaluate results and feedback into decision-making.

4. Poverty analysis, monitoring and evaluation at the four following steps: • Step 1. Policy selection : Situation Description General Advocacy Detailed Analysis • Step 2. Policy elaboration: Setting Goals Setting Targets Selecting Indicators • Step 3. Policy implementation: Monitoring Evaluation • Step 4. Feedback: Transparency and Accountability Improved Interventions and Future Policies

5. What do we mean by poverty? Poverty can be seen as deprivation in both material and non-material goods and assets: • Material poverty: • Income poverty • Consumption poverty • Non-material poverty: • Poor health • Low education or illiteracy • Social exclusion • Insecurity • Lack of freedom and voice/lack of empowerment

6. Goals, Targets, and Indicators: Definitions • GOALS are broad social objectives set in the PRSP documents or MDG declarations in general terms, such as reducing poverty and improving educational attainments. • TARGETS are particular values of goals/objectives that the governments and donors set to achieve by their actions in a specified period of time. • INDICATORS are specific empirical measures required to monitor progress towards achieving a goal (such as proportion of population living below the poverty line) or a target (such as increasing literacy rate by certain percentage).

7. Types of indicators • INPUTS • Resources committed to a project or program. • OUTPUTS • Goods and services produced by a project/program. • OUTCOMES • Variables that policymakers expect to be directly influenced by project/program interventions • IMPACTS • Variables that are likely to be affected by project/program interventions.

8. An example IF GOAL: Increase well-being of rural women. • INPUTS: Financial, human and technical resources. • OUTPUT: Number of rural primary schools, number of trained female teachers in regular attendance, introduction of better gender-balanced textbooks, etc. • OUTCOMES: Higher enrollment rate of girls in primary schools. Higher literacy rate/mean academic achievement scores among girls in rural primary schools. • IMPACTS: Reduced fertility, higher incomes and increased empowerment of rural women.

9. Another example

10. Monitoring MONITORING is concerned with tracking implementation of policies or programs, and with observing how the values of different indicators against stated goals and targets change over time.

11. Monitoring • MONITORING questions may relate to monitoring inputs, outputs, and outcomes: questions may include: • Are the public resources received by health clinics in a given region in line with the amounts agreed in the budget? • Is net primary school enrollment of girls in rural areas rising? • How is consumption-poverty changing over time in urban areas? • Is the perception of the poor about their access to public services changing over time?

12. Why monitor? • MONITORING provides basic information as to whether a program, policy, or development strategy is (i) being implemented as planned and (ii) achieving its objectives. • MONITORING is a crucial tool for four purposes: • Effective management: If actual implementation diverges from planned implementation, monitoring provides evidence of problems that need to be identified and solved. • Policy transparency: Citizens and civil societies are entitled to information concerning policy formulation, financing, and implementation.

13. Why monitor? • Democratic accountability: Citizens are entitled to know whether the actual allocation of public expenditure matches the allocation as promised by the government. If there is a mismatch, they are entitled to know why. • Feasible target setting: It is very difficult to know what is possible to achieve in the future without knowing what has been possible to achieve in the past. It is only possible to know what has been achieved in the past through monitoring.

14. What to monitor? • It is more useful to identify a small number of indicators, measure them well and use the results for policy-making than to enumerate a long list of indicators, measure many of them badly and not use the results for policy decisions at all. • A priority list of outcome indicators might consist of a set of income poverty indices together with some measures of health status and educational attainment.

15. What to monitor? • Once such a list is prepared and disaggregated by sector (rural and urban), region and gender, the number of indicators rises very quickly. • Some countries include a long list of indicators in their PRSPs, e.g., Tanzania has more than 100 indicators listed in its PRSP.

16. How to monitor? • Once targets are set and the indicators selected, the challenge is to manage a monitoring system that integrates a variety of different types of information. • Types of information relevant to such a system may include: • Management Information System (MIS) • Surveys and censuses: Quantitative data • Participatory monitoring exercises: qualitative data • It is useful to draw up a poverty monitoring matrix which identifies the data sources for each indicator, the frequency of measurement and the organization responsible for collecting the information.

17. How to monitor? • Given the wide varieties of information required, it is important to plan ahead so that information is collected in a sequence and maximizes the complementarities between different types of data. • Outputs of the Poverty monitoring System: Annual poverty reduction progress report (required for PRSP process), creation of poverty database, disaggregated spatial poverty map, ad-hoc studies. • It is important that these outputs are actually used to monitor policy and feed into the policymaking process.

18. Monitoring system: What limitations? There are many actors in the information system, performing many activities. BUT, there is typically a series of issues: • Roles badly defined • Lack of coordination between the different actors: duplication, competition, gaps… • Lack of reliability of some information • Difficulty in accessing information • Lack of relevance of some information • Long delays in production of information • Lack of use of the data by the users (don’t know about them, receive them too late, receive a format which is not adapted, lack of confidence, etc.)

19. Monitoring system: What components? Actors: • Data producers • Analysis producers • Data and analysis users, including: • Decision makers • Stakeholders (beneficiaries of services, civil society, media, parliament, etc.) Activities: • Collecting and processing information • Data analysis • Dissemination and feedback

20. Monitoring system: Where to start? • Stock-taking: Who are the actors? What are there activities? What is their capacity? What is their role (officially, in practice), what are the limitations, the barriers? What are their needs? Legislative constraints? • Review the actions currently undertaken to build or reinforce capacity (projects, international initiatives, etc.). • Review of needs in information (in light of the matrix of policies and interventions). • Review of the institutional framework

21. Why evaluation? • IMPACT EVALUATION helps identify the causal link between a policy intervention and an outcome to be identified and measured. In contrast, monitoring helps track down whether a policy or program intervention is being implemented as planned and whether the program or policy is achieving its objectives. Assessing impact indicators is more difficult than monitoring inputs, outputs and outcomes. Monitoring and evaluation (M&E) helps policymakers better design programs/policies and enhance their effectiveness. Better M&E also promotes social debate on poverty reduction strategies.

22. Why Evaluation? • Impact evaluation questions include: § Did a labor training program contribute to job placement? § Did the provision of textbooks contribute to students’ improved test scores? §  Did a micro-finance project improve household welfare? § Did the construction of a rural road improve rural welfare?

23. How to evaluate? • Good practice: • Impact assessment of a program or policy intervention requires a comparison of outcomes before and after intervention with outcomes with and without intervention. The problem is often with finding an appropriate comparison group to identify the counterfactual.

24. What to evaluate? • As impact evaluations are demanding activities, these should be done only if the program intervention warrants an impact assessment. Three questions can help decide when to conduct an impact evaluation. • Is the policy or program considered to be of strategic importance (such as having highest poverty impacts) for poverty reduction? • Will the evaluation of a program or policy contribute to our knowledge gaps of what works and what does not in poverty reduction? • Is the policy or program testing an innovative approach to poverty reduction?

25. Concluding remarks on poverty information systems • Monitoring POVERTY indicators (inputs, outputs, and outcomes) must be well-defined and smaller in numbers. • Both quantitative and qualitative methods may be used to monitor input-output- outcome indicators of interventions • Monitoring is easier and less expensive than evaluating impact indicators • No method of impact assessment is perfect -- conceptualize the problem and use the most appropriate method, given resources and time. • Impact evaluation is more difficult with economy-wide intervention • M&E is a tool for learning whether and how programs matter, and requires planning and resource allocation for its better use.

26. Information Strategy for Effective Poverty Monitoring and Evaluation Development profile Public interventions National Development Strategy Monitoring outcomes Analytical support Demand for information Measurement/Diagnostics Poverty Analysis Monitoring Evaluation Evidence based policymaking Effective Poverty Reduction Accountability Transparency Setting Priorities Supply of Information Legislation Structure Management National Statistical System ResourcesTechnical capacity Supply of data Demand for data

27. Identify capacity building needs • Data management in the MIS of line ministries. • Household survey design and implementation. • Poverty measurement and diagnostics. • Skills for PPAs and participatory monitoring. • Management skills to coordinate overall poverty monitoring and evaluation system • Poverty impact evaluation.