Evaluating Support for Private Sector Development DFID s Perspective Measuring Impact of Business Facilitation on Ent

1. Evaluating Support for Private Sector Development � DFID�s PerspectiveMeasuring Impact of Business Facilitation on Enterprise DevelopmentInternational WorkshopLima, PeruMay 15-16, 2007 Saul Morris Department for International Development

2. OUTLINE This presentation will cover 4 areas: A brief review of recent DFID activities in evaluating PSD interventions A summary of DFID�s Integrated Impact Assessment Approach Current and planned activities in PSD M&E M&E of IFC/DFID collaboration in Latin America

3. Some recent DFID activities in evaluating support for PSD Creation of the Enterprise Development Impact Assessment Information Service (EDIAIS) (www.enterprise-impact.org.uk) (2001) Review of DFID Activities in the business enabling environment (2003) Development of guidelines on the Integrated Impact Assessment (IIAA) Approach to business environment reform (2005) EDIAIS The web-based EDIAIS has been one of our most substantial knowledge development activities in the area of M&E for private sector development support. Designed and managed by Wise Development consultants and the Institute for Development Policy and Management, University of Manchester. EDIAIS has developed and accumulated a huge reservoir of knowledge, tools and case studies on all aspects of M&E and impact assessment for enterprise development. Review of DFID Activities in the Enabling Environment This was an in depth review of about 150 enabling environment projects during the period 1997-2003 period which describes the amount and geographical distribution of DFID funding for different types of enabling environment activities. A key finding of the review was the relatively weak impact assessment carried out at the country project level, making it impossible to estimate overall development impact of spending on PSD support. This was an important factor in the motivation to develop best practice guidelines on impact assessment of PSD interventions, the IIAA method (see next para). The review is available at www���.(to look out). Integrated Impact Assessment Approach DFID decided to try to develop a standardised approach to impact tracking of PSD interventions that would be a practical and useful tool to DFID country offices and the international development community generally. This was developed with the support of the EDIAIS team. The IIAA is not official DFID policy, but is intended to be used as a practical, flexible tool to help develop more robust M&E processes for evaluating business environment interventions. At the country project level in DFID, the main tool used traditionally for M&E is the Logical Framework. In practice, individual DFID projects develop their own impact tracking systems, or conduct one-off impact studies of project performance. There is increasing interest and willingness to spend more time and money on this area, especially as there is a trend for DFID PSD interventions to become larger. Elements of the IIAA approach have recently been applied in several large PSD projects, including in Bangladesh, Sierra Leone and Vietnam. The next two slides summarise key features of the approach.EDIAIS The web-based EDIAIS has been one of our most substantial knowledge development activities in the area of M&E for private sector development support. Designed and managed by Wise Development consultants and the Institute for Development Policy and Management, University of Manchester. EDIAIS has developed and accumulated a huge reservoir of knowledge, tools and case studies on all aspects of M&E and impact assessment for enterprise development. Review of DFID Activities in the Enabling Environment This was an in depth review of about 150 enabling environment projects during the period 1997-2003 period which describes the amount and geographical distribution of DFID funding for different types of enabling environment activities. A key finding of the review was the relatively weak impact assessment carried out at the country project level, making it impossible to estimate overall development impact of spending on PSD support. This was an important factor in the motivation to develop best practice guidelines on impact assessment of PSD interventions, the IIAA method (see next para). The review is available at www���.(to look out). Integrated Impact Assessment Approach DFID decided to try to develop a standardised approach to impact tracking of PSD interventions that would be a practical and useful tool to DFID country offices and the international development community generally. This was developed with the support of the EDIAIS team. The IIAA is not official DFID policy, but is intended to be used as a practical, flexible tool to help develop more robust M&E processes for evaluating business environment interventions. At the country project level in DFID, the main tool used traditionally for M&E is the Logical Framework. In practice, individual DFID projects develop their own impact tracking systems, or conduct one-off impact studies of project performance. There is increasing interest and willingness to spend more time and money on this area, especially as there is a trend for DFID PSD interventions to become larger. Elements of the IIAA approach have recently been applied in several large PSD projects, including in Bangladesh, Sierra Leone and Vietnam. The next two slides summarise key features of the approach.

4. DFID�s Approach: Integrating PSD impact assessment with the Programme Logframe This slide shows diagrammatically the link between DFID�s PSD impact assessment approach and the the project logframe. The top row of boxes shows the four, connected levels of the logframe. The bottom row of boxes translates the standard logframe into a regulatory reform intervention. The first box describes the donor project inputs and activities e.g. TA to simplify business licensing procedures. The second box specifies the output achieved e.g new licensing procedure drafted and enacted. The third box postulates the outcome in terms of the response of firms e.g. increase in business registrations. The last box shows final developmental impact e.g. increased output, incomes and employment.This slide shows diagrammatically the link between DFID�s PSD impact assessment approach and the the project logframe. The top row of boxes shows the four, connected levels of the logframe. The bottom row of boxes translates the standard logframe into a regulatory reform intervention. The first box describes the donor project inputs and activities e.g. TA to simplify business licensing procedures. The second box specifies the output achieved e.g new licensing procedure drafted and enacted. The third box postulates the outcome in terms of the response of firms e.g. increase in business registrations. The last box shows final developmental impact e.g. increased output, incomes and employment.

5. DFID�s Approach: Linking Impact Assessment to the Programme Management Cycle A second feature of DFID�s approach to PSD evaluation is the linking of M&E and impact assessment to the Programme Management Cycle as shown in this slide. The diagram illustrates the four main phases of our impact assessment methodology: (1) initial screening to identify and agree the intervention (2 ) ex ante appraisal at the programme design stage (3) setting up an M&E system for programme implementation and (4) ex post evaluation at the programme completion and review stage. Stage 1: Project concept note stage includes a broad assessment of which regulatory reforms will lead to more impact than others. Stage 2: Ex ante impact appraisal at detailed programme design stage is important to develop the baseline for a sound impact tracking system. Issues addressed include selection of relevant output, outcome and impact indicators, and causal chain analysis showing the projected route from activities to output to outcomes and to impact. Stage 3: DFID has a standard monitoring report procedure based on a dynamic logframe structure and linked to its central MIS system. Reporting is undertaken quarterly, annually, mid tem, and on project completion. Stage 4: The final ex post evaluation stage has two main aspects: comparison of progress against the baseline scenario without the proposed regulatory reform intervention, and comparison of actual results with the projected scenario with the intervention.A second feature of DFID�s approach to PSD evaluation is the linking of M&E and impact assessment to the Programme Management Cycle as shown in this slide. The diagram illustrates the four main phases of our impact assessment methodology: (1) initial screening to identify and agree the intervention (2 ) ex ante appraisal at the programme design stage (3) setting up an M&E system for programme implementation and (4) ex post evaluation at the programme completion and review stage. Stage 1: Project concept note stage includes a broad assessment of which regulatory reforms will lead to more impact than others. Stage 2: Ex ante impact appraisal at detailed programme design stage is important to develop the baseline for a sound impact tracking system. Issues addressed include selection of relevant output, outcome and impact indicators, and causal chain analysis showing the projected route from activities to output to outcomes and to impact. Stage 3: DFID has a standard monitoring report procedure based on a dynamic logframe structure and linked to its central MIS system. Reporting is undertaken quarterly, annually, mid tem, and on project completion. Stage 4: The final ex post evaluation stage has two main aspects: comparison of progress against the baseline scenario without the proposed regulatory reform intervention, and comparison of actual results with the projected scenario with the intervention.

6. Current Knowledge Development Activities in M&E Better Regulation for Growth programme � measuring the quality and impact of regulatory reforms M&E toolkit for business enabling environment technical assistance projects (with IFC and GTZ) Better Regulation for Growth The Better Regulation for Growth programme, which was launched in March 2007, is a two-year joint initiative between DFID, FIAS and The Netherlands aimed at developing and disseminating best practice operational policy guidance that will help developing countries design and implement effective regulatory reform programmes. One of its areas of focus is on measuring regulatory governance. Programme activities will include a review of best-practice M&E frameworks on regulatory quality; regulatory quality and regulatory governance indicators, including baseline measurement methodologies; and regulatory quality standards for developing countries. M&E Toolkit for Business Enabling Environment TA Projects This project, which started earlier this year and will be completed by end 2007, is being led by IFC�s SME Department, in collaboration with DFID and GTZ. The aim is to develop a practical toolkit that will guide project teams working on business environment reform to design and integrate an M&E system into their TA projects. Project activities include research and stock-taking of M&E approaches and experience to date, case studies and field research, survey of donor M&E needs and interests, and preparation of a manual/toolkit. DAI Europe has been contracted by IFC to help develop the toolkit. Better Regulation for Growth The Better Regulation for Growth programme, which was launched in March 2007, is a two-year joint initiative between DFID, FIAS and The Netherlands aimed at developing and disseminating best practice operational policy guidance that will help developing countries design and implement effective regulatory reform programmes. One of its areas of focus is on measuring regulatory governance. Programme activities will include a review of best-practice M&E frameworks on regulatory quality; regulatory quality and regulatory governance indicators, including baseline measurement methodologies; and regulatory quality standards for developing countries. M&E Toolkit for Business Enabling Environment TA Projects This project, which started earlier this year and will be completed by end 2007, is being led by IFC�s SME Department, in collaboration with DFID and GTZ. The aim is to develop a practical toolkit that will guide project teams working on business environment reform to design and integrate an M&E system into their TA projects. Project activities include research and stock-taking of M&E approaches and experience to date, case studies and field research, survey of donor M&E needs and interests, and preparation of a manual/toolkit. DAI Europe has been contracted by IFC to help develop the toolkit.

7. WBG/DFID Collaboration in Enabling Environment in Latin America La Paz Municipal Simplification project- example of best practice S�o Paolo Municipal Simplification project. Expect to collaborate on evaluation Themes?: firm survival rates, incentives for corruption at local government level NE states of Brazil simplification (planned). Themes?: gender and race and SME competitiveness; environmental regulation and competitiveness Central America WBI firm-level survey on publicly financed Business Development Services- consolidated M&E?

8. Thank you!