ESF AND THE ROMA IN SLOVAKIA

1. ESF AND THE ROMA IN SLOVAKIA Results from a study on the territorial distribution of Roma-relevant ESF projects (2007 – 13) and their impact on Roma communities in Slovakia Jakob Hurrle, Charles University Prague; Andrey Ivanov, UNDP BRC

2. 1. Context

3. Focusing on results… • Few important questions: • How to define “result”? How to measure it? • What kind of data is needed? Where to get it? • Most of all, how to move from monitoring the aggregate status of the population to project-level outcome monitoring and evaluation • To assess effectiveness of interventions • To assess the efficiency of funding (ESF and other) • Those questions are part of the Roma pilot project: tools and methods for evaluation and data collection funded by DG Regional Policy

4. Intermediate Final Input Output Outcome Impact Financial, physical resources Goods and services produced by inputs (classrooms built, textbooks provided) Access to, use of, and satisfaction with services (enrolment, repetition, dropout rates) Effect on dimension of well-being (literacy) The M&E chain

5. Example: Employment generation project • Inputs: Number of training courses, expenditures for re-qualification, number of presentations, number of attendees • Outputs: Number of people who passed re-qualification course • Outcome: Number of former unemployed who found jobs • Impact: Registered increase of HH incomes, change in poverty rates Plus • Sustainability: duration of the job • Positive externalities: Reduced drop-our rates of children at risk, reduced societal fragmentation “Intervention”  A  B  C Intervention seen by A is not what is seen by C

6. From theory to practice: ESF • How the structural funds ‘perform’ in reality in regards Roma in the case of Slovakia? • Does the system and the existing procedures allow following the M&E chain? • If not, what is missing? • What can be improved for the next programming period?

7. Assumptions behind the study • Improving the situation of Roma requires a combination of • Political will • Resources • Conceptual framework of intervention • Implementation procedures – the ‘nitty-gritty’ of the project cycle • Usually we focus on (1) and (2), less often on (3) and almost never on (4)

8. A number of specific questions… • Does the territorial distribution of the projects match the distribution of the Roma population? • What share of Slovakia´s Roma population was reached? • To what extend were the invested resources indeed relevant for the needs of the members of marginalized Roma communities? • What are the experiences of project owners and what impact did the projects have on beneficiaries? • Are the project results sustainable? All this fits into the desired “outcome level M&E”

9. 2. Methodology

10. Combination of two approaches: • 2.1 Data analysis 2.2 Case study Prešov Region

11. 2.1 Data analysis Three data sources: • Key data from all „Roma relevant“projects • Sample of 298 Roma-relevant projects created on basis of analysis of project documentation • Data set of Atlas of Roma Communities (2004)

12. Data analysis - key questions: • What does „Roma-relevant“ mean? Method: Analysis of project documents • Which types of projects are „Roma-relevant“? Method: Analysis of project documentation in sample • Which communities where targeted? Method: Matching of project data and data from Atlas of Roma Communities (2004)

13. 2.2 Case study Prešov Region • Focus on training and employment projects in a structurally disadvantaged region with a high share of Roma • 8 projects visited • Attempt to create a sample that represents variety of project types (different types of project owners, ethnic projects / mixed groups of beneficaries, urban / rural, open and closed settings)

14. 3. „Roma-relevant“ projects in the Slovak ESF programme

15. National projects 71 % Out of these Roma-relevant: 8,7 % Calls for proposals 29 % Out of these Roma-relevant: 13,4 %

16. „Roma-relevant“ projects within the five ESF priority axes Axis 1: Supporting Employment Growth Roma-relevant: 22 % Axis 2: Social inclusion Roma-relevance: 86 % Axis 3: Bratislava-region programme Roma-relevance: 35 % Axis 4: Building capacities and improving the quality of the public administration No Roma-relevant projects Axis 5: Technical support

17. Priority axis 1: Employment • 80 projects of total value 32,840,855 €, average size 410,511 € • Project owners tend to be commercial companies • Roma are often just one target group among many (e.g. 1 working place) • Unclear how many targeted but even a project with 1 Roma “ticked” as MRC relevant • Focus often on investments into existing staff – Roma are disadvantaged due to their exclusion from formal labour market

18. Priority axis 2: Social inclusion. 2.1 and 2.2 574 (570) of total value 113,870,162 (48,891,967) € Action 2.1: Social field work (69% or 59%) • 491 (488) projects of total value: 78,951,068 (28,972,874) € • Ethnically defined beneficaries, relative small average budgets sizes 160,796 (59 371)€) Action 2.2: Employment , training (31% or 41%) • 83 (82) projects of total value 34,919,094 (19,919,094) €; Very various groups of beneficaries. Average budget size: 420,712 (242 916)€

19. Territorial aspects a) Does the territorial distribution of the projects match the distribution of the Roma population?

20.  99,2 – 62,2 % 61,9 – 35,6 %  35,4 – 21,6 %  21,4 – 10,7 % 10,4 – 4,3 % 4,3 – 0,01 %

21. Territorial distribution of costs Examples:

22. Features of targeted municipalities (villages only)

23. ESF projects in smaller-than-average villages (sample-based)

24. Level of segregation and unemployment Working index: • Location of settlement • Type of settlement • Distance of the settlement • Physical barrier • Land ownership • Access to electricity • Public light • Garbage collection in Roma settlement Locations are on a rank between 0 (no segregation) to 15 (extreme level of segregation and underdevelopment)

25. Segregation and underdevelopment of targeted municipalities

26. Major findings • A large number of applications is based on almost identical texts suggesting massive copy/paste • The data base doesn’t allow even basic outcome evaluation (outcomes impossible to detect) • Complexity of administrative procedures makes grant owners dependent on (expensive) consultancy services; various forms don’t generate meaningful data • Even though Roma are highly overrepresented among the unemployed, a tiny fraction of projects under priority axis “Supporting Employment Growth” were labeled (on questionable grounds) as “MRC relevant” • Information on the actual ethnic composition of projects´ target groups is missing, which makes it impossible to determine how many Roma are reached

27. The broader context • The current setting does not allow robust outcome evaluation of ESF projects • Number of Roma beneficiaries is vague • Outcomes are not clearly defined and cannot be quantified • Cost-benefit analysis is impossible (only costs/inputs and accounted for) • The system is heavily skewed towards “social work” that is supposed to provide basic social assistance but does little to lift people out of poverty

28. Recommendations • Simple changes: • Modify the application forms so that they generate meaningful data • Applications should be encouraging “focus on outcomes and allow for M&E • Integrate the project level information in processable data management system (currently information is in pdf files) • Modify the TSP workers functions and TORs so that they are part of a local level data generation system for generating project outputs and outcome related data