1 / 12

Doing Impact Evaluations in Argentina and Chile:

Doing Impact Evaluations in Argentina and Chile:. Pragmatism, Opportunities and Partnership. Country Context for Evaluation. Argentina Weaknesses in Survey Data Highly Politicized, Particularly Relations between Central and Lower Levels of Government Lack of openness on information

Jimmy
Download Presentation

Doing Impact Evaluations in Argentina and Chile:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Doing Impact Evaluations in Argentina and Chile: Pragmatism, Opportunities and Partnership

  2. Country Context for Evaluation Argentina • Weaknesses in Survey Data • Highly Politicized, Particularly Relations between Central and Lower Levels of Government • Lack of openness on information • Lack of institutional capacity in area Chile • Evaluation Culture Led by Ministry of Economy

  3. Impact Evaluations 1997-2006 Argentina • Trabajar (workfare) Program • Workfare Participation and Exit • Private Employment Subsidy • Heads of Household Project • Grant-financed Productive Activities Chile • Chile Solidario

  4. Motivations • Serious link with possible follow-up operation (Trabajar) – role of Bank management • Risk mitigation measure (Heads of Household/grants for productive activities) • Government/Bank shared technical interest (private employment subsidy, study of workfare leavers, Chile Solidario)

  5. Pragmatic Design • Used existing surveys (Social Survey, Labor Force Survey, CASEN) • Partnered with local institutions (Ministry of Labor, SIEMPRO, INDEC, MIDEPLAN) • Partnered with DEC staff (Bank supervision budget and other resources) • Government financed out of loan funds • Cost-effective impact evaluations

  6. Pragmatic Design continued • Argentina productive projects: 3rd best design, rigorously implemented • Chile: Based on how program was implemented, recovery from last minute change on sample • Don’t give in on key items – sample size, innovation (psycho-social questions), quality of technical advice on evaluation • Random Assignment sometimes possible

  7. Timing Critical • Trabajar: Board approval (6/97) Prel. Evaluation results (5/98) • HoH: Board approval (1/2003) Prel. Evaluation results (7/2003) Chile Solidario: Information for New Government

  8. Devil is in the Details • Be ready for close supervision and lots of nitty-gritty work • Carefully monitor implementation (samples, field work, questionnaires) -even then things will go wrong • Be ready to explain design and methodology many times • Not a one-shot deal

  9. Pay-off of Sustained Partnership • Building of Relationships (INDEC, DIPRES- Budget Office in Chile) • Building of Relationships with Bank staff (DEC) • Building of Understanding of Methodologies (Ministry of Labor, DIPRES) • Aiming to make it “second nature”

  10. Link with Operations of Programs • How is program being implemented? • What administrative data is available? • Knowledge/cooperation of operational staff • Important for following up on results • Add value to evaluation • Best partners for impact evaluation are program operators (Ministry of Labor, Chile Solidario/FOSIS- Social Fund and main operators of Program)

  11. Use of Results • Trabajar – Justified follow-on operation • Head of Household – Program credibility, provided information on implementation • Chile Solidario – too soon to tell • “Public goods” contribution • Spill-over effects (use of data by others)

  12. …but, Not a Magic Bullet • Decisions on programs based on many factors, not just their performance • Not everyone convinced by evidence-based data vs. anecdotes or ideology or initial views • Need for better dissemination, particularly in-country • Bank value-added, including for middle-income countries

More Related