Monitoring and Evaluation, Poverty Assessment, Targets and Indicators - An Introduction to the Challenges and Issues in

1. Monitoring and Evaluation, Poverty Assessment, Targets and Indicators - An Introduction to the Challenges and Issues in the PRSP Chris Pain - June 2002 Consultant, GTZ Africa Department, Policy Section Joint Donor Staff Training Activity - Partnership for Poverty Reduction Tanzania, June 17-19, 2002 M&E, Poverty Assessment, Targets and Indicators in the PRSP

2. Presentation Structure Monitoring the PRS (Selection of Indicators) Poverty Assessment and Prioritisation Monitoring and Evaluation - Challenges and Issues in the PRS Capacity Building for Monitoring Participation in PRS Monitoring Dissemination of the Monitoring Results M&E, Poverty Assessment, Targets and Indicators in the PRSP

3. Introduction - The PRS Implementation Cycle M&E, Poverty Assessment, Targets and Indicators in the PRSP

4. Poverty Assessment andPrioritisation M&E, Poverty Assessment, Targets and Indicators in the PRSP

5. Poverty Assessments • Poverty - Dominant Consensus on what the concept means - not just in international definitions, but reflected in PRSs. • This prompts the question whether specific national determinants of poverty are being overlooked • Poverty Assessments - diagnostic processes that identify the extent, causes and trends in poverty • An important mechanism for turning poverty issues into policy • but to do this requires an understanding and acceptance of who the poor are and what poverty is • the determinants of poverty need to be specified for particular groups and regions (Disaggregation) • Contribute to assessing the effectiveness of current policies and whether the poverty situation is changing M&E, Poverty Assessment, Targets and Indicators in the PRSP

6. Poverty Assessments • Progress towards better poverty data is being made but PAs need to • establish policy relevant groups of households and be clear in terms of the actual interventions which would assist these groups • link specific socio-economic groups to the growth and development process • identify the relationship between poverty and household demographic characteristics • improve on the ex ante and ex post poverty and social impact assessment of policy interventions • improve the quality of the data they use M&E, Poverty Assessment, Targets and Indicators in the PRSP

7. Poverty Assessments - PSIA • The increased attention to Poverty Reduction highlights the need for an understanding of how policies affect the poor • A PSIA needs to take a multi-disciplinary approach • use economic and social tools • use quantitative and qualitative methods • Weaknesses in utilising a PSIA in the design of government policy • PSIA is not undertaken early enough to inform the design of policy • The analysis is not made public • The risks of policy implementation are not generally addressed • Potential losers are not explicitly identified • Challenges in Implementing a PSIA • Data Constraints • Analytical Constraints • Capacity Constraints • Time Constraints M&E, Poverty Assessment, Targets and Indicators in the PRSP

8. Poverty Assessment and Prioritisation • A linear approach to policy making suggests after the Poverty Assessment priorities can be established - Evidence Suggest this is not the case • Prioritisation is a complex issue involving more than just “good information” - it depends on • The particular version of poverty that is used • Whose voice is heard • There is a strong political element to such negotiations. An important lesson is that Politics Matters • Lack of Prioritisation leads to Wish Lists that are expressed in broad and evasive terms • There is also a need to prioritise within sectors as well as between them • Participatory Process can help with this - some evidence to suggest that this is being done (eg Mozambique and Tanzania) M&E, Poverty Assessment, Targets and Indicators in the PRSP

9. Poverty Assessment, Consultations and Priority Setting - Proposed Sequence • Disaggregation of “the poor” into specific categories • Analysis of information based on the identified categories • Analysis of past and present poverty reduction programmes with regard to who has been reached • Give Priority to specific categories of households • Develop Specific Strategies for each prioritised category of household • Reconcile priority setting with available resources (cost the options) • Establish targeting mechanisms to ensure the prioritised categories of households are reached M&E, Poverty Assessment, Targets and Indicators in the PRSP

10. Monitoring the Poverty Reduction Strategy M&E, Poverty Assessment, Targets and Indicators in the PRSP

11. Monitoring the PRS • Why Monitor • Track Progress in achieving the poverty reduction goals • Helps to validate the choices made in the first place, justifying government action to the public • Reveal reasons for success or failure, allowing effective management of the strategy and improvements to be made • Mobilises and Sustains Public Support for the Targets • Offers the opportunity for greater involvement of Civil Society in the process • Acts as a means of accountability in the use of resources, generating transparency • In general, monitoring allows for the opportunity to have quality management within the PRS • To Date Monitoring is the Poor Relation in the PRS - it is generally the last thing to be addressed • Poverty Monitoring is not the same as PRS Monitoring • Poverty Monitoring Generally Refers to final indicators • PRS Monitoring is much broader - including expenditure and outputs M&E, Poverty Assessment, Targets and Indicators in the PRSP

12. Monitoring the PRS - Some General Concepts • Goals the objectives a country or a society want to achieve • Indicatorsthe variables used to measure progress towards the goals • Targets the quantified level of the indicators that a country wants to achieve in a given time frame. M&E, Poverty Assessment, Targets and Indicators in the PRSP

13. Monitoring the PRS - Long Term Indicators - The MDGs M&E, Poverty Assessment, Targets and Indicators in the PRSP

14. Monitoring the PRS - Long Term Indicators - The MDGs M&E, Poverty Assessment, Targets and Indicators in the PRSP

15. Monitoring the PRS - Types of Indicators Intermediate Indicators Input Indicators Financial and Physical Indicators (Including Personnel) of Resources Used Output Indicators The Intermediate Goods and Services Generated Access and Use of Goods and Services and Satisfaction of Beneficiaries Final Indicators Outcome Indicators Effect on Key Dimensions of Well-Being (Improvements in Living Standards) Impact Indicators M&E, Poverty Assessment, Targets and Indicators in the PRSP

16. Monitoring the PRS - When is it Useful to Use These Indicators Useful in the Short Run - Source of Information - Budget Documents and Actual Expenditure Data Intermediate Indicators Input Indicators Money Allocated to Primary Education Useful in Short and Medium Term Source of Info - Admin Systems, MIS Number of Schools Built Output Indicators Useful in Medium to Long Terms Source - Priority and Quick Surveys,PPAs Final Indicators Enrolment Rate of Girls Outcome Indicators Useful in Long Run - Source - Household Income Surveys Impact Indicators Literacy Level M&E, Poverty Assessment, Targets and Indicators in the PRSP

17. Policy Reforms Public ActionChoices Public Services Investment Programs Institutional Capacity Building Inputs Outputs Outcome Poverty Monitoring (MDGs, other sectoral outcomes) Impact Monitoring PRS Implementation - A Framework M&E, Poverty Assessment, Targets and Indicators in the PRSP

18. Monitoring the PRS - Experience to Date in Establishing Indicators • Long Term Targets Have Been Adopted • But the inclusion of Good Intermediate Indicators has been neglected • Why has this been the Case • Agreeing what happens in the middle is always the hardest • They are less contentious • The emphasis placed on them in the documentation may have encouraged this • It is now very important to address the issue of the Missing Middle to assure the quality management of the PRS • Clear link between agreed targets and activities - what is needed is not a greater general emphasis on some particular point on the chain from inputs to outputs but greater linkages along the chain • Make it useful and worthwhile to produce and use the information - provide incentives to use the information, eg better access to resources M&E, Poverty Assessment, Targets and Indicators in the PRSP

19. Monitoring the PRS - Other Issues Identified to Date • In general the PRSPs contain too many indicators • This means it would be difficult to monitor. • PRSPs need to contain a small number of realistic core indicators and targets • Prioritisation of Interventions will lead to prioritisation of indicators - needs to involve stakeholders • The Indicators developed to date are linked to the MDGs • They need to be disaggregated to take account of regional aspects • They also need to be more realistic in view of prior achievements and available resources • Over Reliance in the Indicators of monetary poverty reduction and education and health • Now Need to incorporate issues of governance, natural resource management, security and empowerment • Emphasis on technical process of poverty data collection and analysis focus on poverty monitoring outcome and impact level • Statistical information gathering and analysis - risk of not tying into policy / decision making M&E, Poverty Assessment, Targets and Indicators in the PRSP

20. Monitoring the PRS - How is the Information Collected • Traditional Reliance on IHS and WMS supplemented with PPAs • These are bulky, take time and tend to focus on final poverty outcomes • Fortunately there is a large range of survey and non-survey instruments available - • Budget Monitoring includes improving the quality of expenditure tracking, developing medium term perspectives to budget making (such as MTEFs), focusing on performance and promoting greater participation • Management Information Systems offer a huge potential but have an incentive to distort the results and are not trained data collectors • Alternative Data Collection Methodsparticularly relevant to get a quick feedback • CWIQ - Mali, Mozambique, Malawi • Report Cards - Phillipines and India • In the PRS, each have their own strengths and weaknesses. They need to be integrated in a manner that allows them draw on each other. M&E, Poverty Assessment, Targets and Indicators in the PRSP

21. Monitoring the PRS - Impact Evaluation • Evaluation is complementary to Monitoring - coming at the end of the life of a PRS it is important to judge which public actions have been effective and which have not worked • Used to Inform decisions on expansion, modification or elimination of a particular policy or programme. • Key Questions in an Impact Evaluation • Have the interventions achieved their goals • Can changes in the Poverty Situation be attributed to the interventions or have other factors caused the changes • What have been the impacts on different groups - men, women, various ethnic groups • Have there been unintended side effects • How effective are the programmes when compared to alternatives • Are the interventions worth the cost • Data Requirements • Different Sources - Qualitative and Quantitative M&E, Poverty Assessment, Targets and Indicators in the PRSP

22. Participation in PRS Monitoring M&E, Poverty Assessment, Targets and Indicators in the PRSP

23. Participation in Monitoring - Experience of Participation to date in the PRS Framework • so far efforts focussed on participation in the formulation of the strategies • ahead: how to • make it more organic • maintain participation in implementation and in M&E? Participation in PRS Monitoring • a building block to institutionalize participation • … how stakeholders share influence and control over the monitoring process, its results and the subsequent policy and decision making M&E, Poverty Assessment, Targets and Indicators in the PRSP

24. Participation in PRS Monitoring - Why? • moving out of the exclusive circle of MoF and some sector Ministries • bring PRS and its implementation into public domain/ public debate • amplify voice and agency of the weak and usually unheard • increase accountability and transparency of public actions • Increased public awareness by demystifying policies, budgets • contribution to more inclusive public policy debate • Better and more complete information for decision making • direct feedback from citizen • consultation with multiple perspectives • representation of interests (winners/ losers) • Greater transparency and public accountability • Contribute to performance and client orientation of public sector M&E, Poverty Assessment, Targets and Indicators in the PRSP

25. Policy Reforms Public ActionChoices Public Services Investment Programs Institutional Capacity Building Inputs Outputs Outcome Impact Participation in PRS Monitoring - Where Does it Fit? Participatory Expenditure Tracking Citizen Report Cards Qualitative Policy Impact Monitoring, PPA’s M&E, Poverty Assessment, Targets and Indicators in the PRSP

26. Disseminating the Results of PRS Monitoring M&E, Poverty Assessment, Targets and Indicators in the PRSP

27. Disseminating the Results of Monitoring - Why? • Providing Information on the Results of Exercises and Progress towards achieving Goals and Targets can mobilise and sustain public support for the targets • Results that are not disseminated and targeted at specific groups in civil society will not be used. • Resources used in generating information that is not used are wasted. • For Civil Society to Participate Effectively in Issues of the PRSP they must have access to information • Provision of the results to Civil Society improves the general feeling of Transparency and Accountability M&E, Poverty Assessment, Targets and Indicators in the PRSP

28. Disseminating the Results - How? • Have Mechanisms tailored to different PRS Stakeholders • policy makers • parliamentarians • programme managers • civil society organisations • general public • Donors • Utilise the Media to Stimulate Public Debate and enhance accountability of development agencies M&E, Poverty Assessment, Targets and Indicators in the PRSP

29. Disseminating the Results - Generating Demand • Common Complaint in a Number of Countries that the Demand for Information is Generated by Donor Agencies • Need to generate domestic demand for the results • Amongst CSOs - to create pressure for policy change and enhanced transparency • Amongst Government Ministries - Tie Budget Allocation to Performance by Adopting Target Oriented MTEFs • It is only once the results of monitoring initiatives are converted into the public domain can it be expected that action will be taken and therefore can it be said that monitoring has influenced decision making. M&E, Poverty Assessment, Targets and Indicators in the PRSP

30. Capacity Building forPRS Monitoring M&E, Poverty Assessment, Targets and Indicators in the PRSP

31. Capacity Building for PRS Monitoring The quality of country systems for measuring and monitoring results is important for the challenge before us. This puts a premium on our capacity building support for public sector management, statistical development, and monitoring and evaluation systems, which are important in their own right for underpinning countries accountabilities to their people and results agreements and compacts with donors Joint Statement on “Measuring, Monitoring and Managing for Development Results” made by the heads of five development banks at the UN International Conference on Financing for Development The Key Questions • What is the Current Capacity to Monitor the PRS (Where are we?) • What is to be achieved through “Capacity Building” (Where do we want to be?) • What is to be done to improve the capacity (How do we get there?) M&E, Poverty Assessment, Targets and Indicators in the PRSP

32. Capacity Building for PRS Monitoring - Experience to Date • PRSs generally identify the need for capacity development to assist in the Implementation of the M&E System. • It is not clear to what extent assessment of the capacity to monitor the PRS have been carried out • Nor is it made particularly clear where we want to be with this capacity. • They do not assess why capacity previously developed is no longer available. • Further, Capacity for Whom needs to be addressed - must include elements of the public sector, civil society and the private sector have to be included. • Monitoring of the PRS involves the participation of a number of agencies inside and outside government. • PARIS-21 initiative established in 1999 to promote a culture of “evidence based policy-making and monitoring in all countries” M&E, Poverty Assessment, Targets and Indicators in the PRSP

33. PRS monitoring - Implications for Donors • Objective of the OECD Task Force on Donor Practices - Alleviate the cost for partner countries of meeting multiple donor reporting and monitoring requirements • The Monitoring System and Review Process should meet the monitoring and reporting needs of both the Government and Donors • A strong Monitoring System will help to create transparency • Some movement towards this, but there are constraints still • Fear attached to subordinating individual reporting requirements to a common framework • The need to reduce their own reporting and monitoring requirements to the necessary minimum • National Monitoring Systems often lack capacity - this needs to be built M&E, Poverty Assessment, Targets and Indicators in the PRSP

34. Thank you for Your Time and Attention Any Questions? M&E, Poverty Assessment, Targets and Indicators in the PRSP