70 likes | 172 Views
Do we know how to create jobs?. Evaluation lessons from a S ystematic R eview by Michael Grimm and Anna Luisa Paffhausen, University of Passau, Germany C ommi ssioned by KfW Entwicklungsbank Evaluation Unit.
E N D
Do we know how to create jobs? Evaluation lessons from a Systematic Review by Michael Grimm and Anna Luisa Paffhausen, University of Passau, GermanyCommissioned by KfW Entwicklungsbank Evaluation Unit Disclaimer: The views and opinions expressed in the Systematic Review are those of the authors and do not necessarily reflect the official policy or position of any agency of the German Development Cooperation.
Systematic reviews ...a waytoovercometheproblemof limited externalvalidityof experimental (RCT) and quasi-experimental outcome/impactmeasurements. Disclaimer: The views and opinions expressed in the Systematic Review are those of the authors and do not necessarily reflect the official policy or position of any agency of the German Development Cooperation. OECD DAC EvalNet 12-13 February 2014, Paris KfW presents itself / November 2013
84 studies excluded 5 not obtained 15 studies ongoing 1,785 records excluded (mostly because not related to our research question) Search strategy and intervention categories Identifi-cation 2,275 record identified through database searching Screening 1,924 records screened after duplicates removed 20 additional records identified through other sources Eligibility 139 studies assessed for eligibility 55 studies included Incl. 27 RCT studies 93 impact estimates Included Not adding up to 55 because some studies cover more than one intervention type Access to finance and insurance • 26 studies / 13 RCTs Entrepreneurship training • 20 studies / 16 RCTs Intervention types Business development services and targeted subsidies • 10 studies / 1 RCT Improvements of businessenviron-ment: Incentives to formalise • 5 studies / 1 RCT OECD DAC EvalNet 12-13 February 2014, Paris KfW presents itself / November 2013
Key findings on employment • 26 studies, 40 Treatment effects: 13 positive, 2 negative, 25 insignificant • No large employmenteffects, particularly not in micro-enterprise. • More businesscreationthanhiringofnewworkers • Interventionsmostly do not havejobcreationasmajorobjective (but ratherincomestabilisiation, higherfamilyincome). • Employmenteffects larger ifowneris male. Access to finance and insurance • 20 studies, 31 treatmenteffects: 9 positive, 1 negative, 21 insignificant • Successfultrainingdoes not necessarilyleadtoemploymentgeneration: itcanevenleadtoclosing down unsuccessfulenterprise. • Training shouldbe substantial (about 1 year, once a week) andspecific. • Tackling financeimpedimentsatthe same time seemstomakeemploymenteffectsmorelikely. • Entrepreneur-ship training • 10 studies, 7 positive ormixed, 1 negative (minimum wage), 1 insignificant • Targetedsubsidiesforhiring a workerorforinnovations/R&D cancontributetoemploymentgeneration • Subsidiesandservicesshouldbedemand (not supply) driven, tailor-made andfocussed. • Expensive interventions • Business develop-ment servicesand targetedsubsidies • 5 studies, employmenteffect after formalisation 4 positive, 1 insignificant • Onlyfew (larger) andnewfirmsformalise; formicroentrepreneursoftenformalityhasmoredisadvantagesthanadvantages. • Formalisation, whereitworked, hadonlymodestemploymenteffects. • Betterservicesthatfirmsgainaccesstowithformalisationofferincentivestogiveupinformalityandmightcontributetojobcreationaswell. Businessenviron-ment: Incentives to formalise OECD DAC EvalNet 12-13 February 2014, Paris KfW presents itself / November 2013
The method bias • Do quasi-experimental studiesover-estimateemploymenteffects due toun-eliminatedbiases? • ...orareemploymenteffectssmall in RCT-measurementsbecausethismethodisveryoftenappliedtorathersmallprogrammes in relativelypoorareas? OECD DAC EvalNet 12-13 February 2014, Paris KfW presents itself / November 2013
Implications for development policy, evaluation and research Policy • Itis a longwayfrompolicyinputstoemploymentimpacts: itseemseasiertoachieveeffects on managementpractices, salesorprofitsthanemploymenteffects. • A major push isneededtohaveemploymentimpact. • Manyoftheinterventionsincluded in thereviewfirstof all striveforincomestabilisationandpovertyreduction, and not foremploymentcreation. • Targetingseemstobekey. • Itseemstobeeasiertocreatenewbusinessesthantofosterthegrowthofexistingfirms. Evaluation andresearch • Evidenceis still sketchy, particularlyfor Sub-SaharanAfricaandAsia. • Thereisalmost a complete lack ofevidence on long-term effectsandcost-effectiveness. • A dilemma: Fortheinclusioninto a systematicreview, studieshavetomeethighestmethodologicalqualitystandards. However, not all programmesaresuitedfor (quasi-)experimental evaluationdesigns • Howto deal withthedilemma: • Do not neglectthefindingsofothertypesofevaluations • ...untilresearchcomesupwithnewqualitystandardsthatcover a wider rangeofprogrammedesigns OECD DAC EvalNet 12-13 February 2014, Paris KfW presents itself / November 2013
Thank you! OECD DAC EvalNet 12-13 February 2014, Paris KfW presents itself / November 2013