Evaluation of Active Labour Market Measures and Entrepreneurship in Poland

1. Active labour market measures and entrepreneurship in Poland Rafał Trzciński Impact Evaluation Spring School Hungary, 18.05.2011

2. case 1 – Evaluation of @lternative II project • Objective of theproject: preventing unemployment among young people. • Theterritorialscope: 59 poviats of Poland (NUTS 4) withunemploymentrateabove 20% and high unemploymentamongyoungpeople. • Eligibility: • youngunemployed (27 oryounger), • registered at the labour office. • Total number of beneficiaries: 5 657. • Budget: € 4 090 702. • Type of services:pre-training assistance (recruitment, needs assessment, guidance); vocational training services linked with ECDL, both at basic and advanced level, as well as related areas; practical training (temporary employment/ on the job training organised under the agreements signed with employers; training allowances; vocational guidance and post training assistance; job-broking. • Period of implementation: 2005-2006.

3. Evaluation problem SELECTION BIAS Selectionto the project Factor x1 Efect: lower unemploy-ment Self-selection to the project ? Factor xn @lternative IIproject Problem: High unemployment rate among young people

4. The selectionbias problem in control/comparison group approach • To estimate the impact of the intervention we cannot simply compare beneficiaries (treated) with those who did not participate in the project (non-treated) • Thisisbecause of factors affecting both participation and outcomes. • If we don't control for thosefactors, we can overestimate or underestimate the impacts(pickingthewinners/ pickingthelosers).

5. Data used in the evaluation PULS System, which: • isused for services for the unemployed • ispresent in approximately 90% of PoviatLabourOffices in Poland (2006) • provides a wide range of data on eachunemployed person (socio-demographics, employmentcharacteristics, previousqualificationimprovement, skills etc. ), • includes a detailed history of unemployment and otheractivities on each person (registrationintheoffice, deregistration, trainings, use of thebenefits, etc.).

6. Data collection • We collected data from 55 of the 59 Poviat Labour Offices involved in the project. • In total we managed to identify 5 065 participants of the @lternative II project (90% of all beneficiaries). • Moreover we collected data on 126 633 persons (non-treated), which meet the formal conditions for eligibility for the project (registration in the labour office, age condition).

7. Variables • Socio-demographiccharacteristics • Sex • Age • Marital status • Single parenting • Number of children • Education • Poviat • Employmentcharacteristics • Profession (ten categories) • Number of days of work • Number of professions(in total) • Number of days beingunemployed before participating in the project • Number of daysreceivingtheunemployment benefit before… • Number of job offers during the one year period before… • Number of daysparticipatinginsubsidisedwork • Number of days of permanent unemployment (duringthetwoyears period before…) • Previousqualificationimprovement • Number of training courses, in which the person participatedduring the one year prior to participation in theproject • Total number of days spent on training • Having a work placement before participating in the project • Motivation to find a job • Percentage of showing up in the PoviatLabour Office, • Having the right to unemployment benefit • Skills • Possession of drivinglicense(B category)

8. Back to theselection problem…

9. Bearinginmindtheassumptions… ConditionalIndependence Assumption Treatment Counterfactualaction Population A Counterfactualaction Treatment Population B • We assume that if we can control for observable differences in characteristics between the beneficiares andnon-treated population, the outcome (observable change) that would result in the absence of treatment (counterfactual action) is thesame in both populations. • Ergo, we assume that unobservables do not affect the outcomes!

10. Propensityscorematching (1-1; nearestneighbour) Beneficiaries(N=5 065) Eligible non participants (N=126 633) Control group(N=5 065) ps= 0,8 ps= 0,1 ps= 0,5 ps= 0,4 ps= 0,9 ps= 0,6 ps= 0,2 ps= 0,1 ps= 0,3 ps= 0,2 ps= 0,01 ps= 0,8 ps= 0,3 ps= 0,9 ps= 0,4

11. What we have achieved using PSM?

12. Impact Source: Ex-post evaluation of Phare 2003 Economic and Social Cohesion – HumanResources Development component, PAED, Warsaw 2007

13. Impact

14. Cost-benefitanalysis

15. (Counter)example 2 – Entrepreneurshippromotionproject • Objective of theproject:encouraging business activitiesamongunemployedpeople. • Beneficiaries: unemployedpeople (with priority to young job-seekers). • Type of services: initial business training; guidance on conducting economic activities; training allowance; relevant specialised training; coaching after setting up a business. • Time of implementation: 2004-2005. • Evaluation framework: the same approach as in the @lternativa II exaple (the same methodology, source of data, analysis...).

16. Impact? Source: Ex-post evaluation of Phare 2002 Economic and Social Cohesion – HumanResources Development component, PAED, Warsaw 2006

17. Lessonslearned/points for the discussion • What data we werelackinginbothexamples? • Missingcovariates? (Areourassumptionsplausible?) • Missingoutcomevariables? • What do we know and what we don't knowaftercompletingtheevaluation (towardstheorybasedimpactevaluation)? • How we couldmodifythe plan of theevaluaton to getmoreinsight on impacts (targetingissue)? • Whatistheavaibility of systems such as PULS inother EU countries (looking for possibilities of implementing IE)? • Whatistheutility of data collectedin public statistics? Do we neednew data systems for IE ormaybe we need to modifyexistingones? (towardsmoresystematicdiscussion on IE planning).

18. Thank you!!!