Part I: Impact Evaluation of Rural Roads Dominique van de Walle DECRG, October 2006

1. Part I: Impact Evaluation of Rural RoadsDominique van de WalleDECRG, October 2006

2. With limited redistributive instruments, we look to sectors such as rural roads to achieve distributional objectives • General belief that roads are good for development & living standards • But, little is known about impacts

3. What Is Different About Roads? • Impacts are indirect: roads are an intermediate service; impacts depend on interactions w/other investments, household & community factors Important to control for heterogeneity of other factors • But, impacts can be economy wide: potential controls may have been determined by road investment • Roads are not randomly placed

4. Therefore: • Traditional methods of conducting evaluation may be contaminated by the project, directly or indirectly • Potential for randomization is limited • Need: • baseline and panel • appropriate controls for exogenous time varying factors i.e. double difference + controls for observed time varying factors

5. Getting at distributional impacts: • On average benefits may be positive • But, there may be both gainers & losers • Short & longer term distributional impacts may be different • So need: • ability to differentiate between welfare groups • sufficiently long post-project follow-up

6. Learning from the Ex-Ante Evaluation Think about evaluation from day 1 of project ‘Appraisal’ stage can be thought of as ‘ex-ante evaluation’ How are intervention areas picked? How are road links chosen? Helps to understand biases in ex-post evaluation

7. Part II: Application to Rural transport project (RTPI) in Vietnam • Rural roads rehabilitation project in 18 provinces of Vietnam, 1997-2001 • To link communities with markets and reduce poverty • Rehabilitation of rural roads; no new roads are to be built. • Selection criteria: costs less than $15,000 per km; population served is at least 300 per km.

8. Evaluation Questions • What are the impacts of rural roads on living standards (broadly defined) and their distribution? • What factors influence those outcomes? • How much do benefits depend on other investments (e.g. human capital)? • In what ways do first-round impacts differ from longer term impacts?

9. Survey of Rural Road Impacts in Vietnam (SIRRV) • Pre-project baseline in 1997 • Post-project follow-up rounds every two years: 1999, 2001 and 2003. • 100 “treatment,” 100 “comparison” communes in 6 provinces; randomly chosen • 15 households per commune: stratified sampling • Panel of 200 communes & 3000 households. District and project data bases • No welfare indicator; possibility of linking up with VLSS98

10. Risks • timing: project delays • commune splits • weather: typhoons, flooding etc • waning interest • fungibility: project aid displacement

11. Evaluation methodology Diff-in-diff with propensity score matching • Propensity-score matching • Using baseline characteristics likely to affect selection into the project, and outcomes • This deals with observable heterogeneity in initial conditions that can influence subsequent changes over time • Diff-in-diff • Difference in outcomes over time between matched project and non-project communes • Purges additive time-invariant unobservables

12. Evaluation methodology • Logit model of program participation on pooled project and non-project samples. • Create predicted values from the logit regression – the propensity scores • Check for common support • Nonparametric kernel matching is used for one estimator • An alternative estimator uses the propensity scores to weight the DD

13. Impacts on rehabilitated road kms? • Impacts on rehabilitated road km were less than intended • However, more roads were built in project areas. • Thus, we find fungibility within the sector, but evidence of a flypaper effect of aid on roads • Spending on rehab + building accords reasonably closely to total amount allocated by the project

14. Other findings so far: • Impacts on market development, availability of goods, household involvement with markets • But, very heterogeneous impacts across communes, depending on initial conditions.