European Cohesion Policy – European Social Fund Monitoring and Evaluation in 2014-2020

1. European Cohesion Policy – European Social Fund Monitoring and Evaluationin 2014-2020 Effie Meletiou Impact Assessment and Evaluation DG Employment, Social Affairs and Inclusion Nicosia, 26 November 2013

2. Outline • Monitoring: highlights of regulatory requirements • Monitoring: regulatory requirements on indicators • Monitoring: programme specific indicators • Evaluation

3. Monitoring: Highlights of regulatory requirements (with comments)

4. Monitoring committee (Art. 41) • Set up within 3 months of adopting the OP • Single MC can cover more than one programme • MC draws up its own rules of procedure with reference to institutional, legal and financial framework of the MS concerned

5. Composition of MC(Art. 42) • Decided by MS provided that it is composed of relevant MS authorities, IBs and partners referred to in Art. 5. • Representatives of these partners shall be delegated through transparent processes. • Each MC member may have a voting right • The list of members shall be published

6. Functions of the MC (Art. 43) MC shall meet at least once a year Review implementation of OP(s) Shall be informed of progress of achieving targets and milestones in the performance framework and results of qualitative analyses Shall examine all issues affecting performance of the programme, including the conclusions of the performance review Shall be consulted on OP modifications and give opinion, if it considers it appropriate May make observations regarding OP implementation and evaluations, including actions related to reduction of admin burden on beneficiaries Regional Policy Cohesion Policy

7. Functions of the MC (Art. 100) Shall examine: Issues affecting performance Implementation of evaluation plan and follow-up given to evaluation findings, Implementation to communication plan, JAPs & financial instruments Progress in fulfilling applicable ex ante conditionalities, where not fulfilled at the time of submission of OP or PA Shall examine and approve: Evaluation plan (covering one or several OPs) and any modification thereof Communication plan Modification of OP Regional Policy Cohesion Policy

8. Important changes in Regulations, Implementing and Delegated Acts • Enhanced focus on results • Increased importance of monitoring and evaluation • Even stronger need for clear intervention logic • Close link with Europe 2020 strategy

9. Clear intervention logic • Composed of the hierarchy of programme objectives, actions, expected outputs and results • Important for all phases: programme design, implementation, monitoring and evaluation • E.g. choice of programme-specific indicators, time-planning of evaluations

10. Clear intervention logic • Challenges and needs • Country-specific recommendations • National Reform Programme • European semester analyses • Consistent translation into thematic objectives and investment priorities Funding priorities Linkage to Europe 2020 Strategy

11. Programme theory – Intervention logic

12. Programme architecture Thematic objective Priority axis Investment priority 1 Investment priority 2 Specific objective 1 Specific objective 2 Specific objective 1 Specific objective 2

14. Programming (1) • Identification of development needs • Selection of thematic objectives and investment priorities as set out in the CPR and Fund-specific rules • Definition of at least one specific objective per investment priorityto target the latter appropriately in the specific national or regional context • Definition ofresultindicators linked to specific objectives

15. Programme architecture Thematic objective Priority axis Investment priority 1 Investment priority 2 Specific objective 1 Specific objective 2 Specific objective 1 Specific objective 2

16. Specific objectives Art. 2 CPR - Definitions "specific objective' means the aim to which an investment priority or Union priority contributes in a specific national or regional context through actions or measures undertaken within such a priority"

17. Importance of specific objectives Art. 17 CPR – ex ante conditionalities 'applicable ex ante conditionality' means a concrete and precisely pre-defined critical factor, which is a prerequisite for and has a direct and genuine link to, and direct impact on, the effective and efficient achievement of a specific objective for an investment priority or a Union priority

18. Importance of specific objectives Art. 87(2)(b) CPR – Operational Programme For each priority axis: "the investment priorities and corresponding specific objectives" "in order to strengthen the result-orientation of the programming, the expected results for the specific objectives, and the corresponding result indicators, with a baseline value and a target value, where appropriate quantified in accordance with the Fund-specific rules" "a description of the type and examples of actions to be supported under each investment priority and their expected contribution to the specific objectives"

19. Importance of specific objectives Annex II – performance framework "Milestones are intermediate targets, directly linked to the achievement of the specific objective of a priority, where appropriate, expressing the intended progress towards the targets set for the end of the period" Milestones and targets shall be "consistent with the nature and character of the specific objectives of the priority"

20. Equal opportunities Art. 7 Equal opportunities "The Member States and the Commission shall ensure that equality between men and women and the integration of gender perspective are taken into account and promoted throughout the preparation and implementation of programmes, in relation to monitoring, reporting and evaluation."

21. Programming (2) • Onlyafterthe development needs, objectives and the result sought have been clarified, should one consider the types of actions to be supported, choosing (a mix of actions) which best achieves the objectives defined • Output indicators should be logically linked to and reflect the types of actions planned. Outputs generated by actions should also contribute logically towards the results that one aims to achieve.

23. Programming (3) • Completing the drafting of the intervention logic • Reflection and adjustment of intervention logic • Testing the intervention logic – ex ante evaluation • Reflection and adjustment of intervention logic

24. General remarks on indicators • Common and where relevant programme-specific • Financial, outputand result • No impact indicators (difference with 2007-2013) • Reported annually andelectronically by MA as structured data as part of the AIR, broken down by investment priority • Reported as annual data, not cumulatively (difference with 2007-2013) • Relate to partially or fully implemented operations (definition in Art. 2 draft CPR)

25. General remarks on indicators • Recommendation: monitoring data entered into system throughout the year by beneficiaries/ bodies in charge of entering monitoring data • Clear name, unequivocal and easy to understand definition, measurement unit (provided for common indicators)

26. Monitoring information system The system must record and store data on individual participants in a way that permits the managing authorities to perform the tasks related to monitoring and evaluation in conformity with the requirements set out in Art. 49 and Annex XX CPR and Articles 5 and 15(iv)(4) and (6), Annex I and II of the ESF Regulation Tasks include: • undertake impact evaluations • Be able to contact participants after they have left the support • draw a representative sample of participants

27. Micro-data • CPR (Art. 114(2)(d)) sets out a legal obligation for the managing authorities to establish a system that records and stores individual participant data in computerised form • Micro-data of participants should be collected and stored • Micro-data are observation data collected on an individual object, i.e. a participation record • Observation data (characteristics and results) collected by indicators • Micro data allow MS to create output/result statistics and to match different observation data • May be complemented by unique personal identifiers

28. Data protection The data processing arrangements must be in line with the provisions of Data protection Directive 95/46, in particular Articles 7 and 8 thereof. Indicators require the collection of two categories of data on individuals: • Personal data – indicator marked with * • Personal sensitive data – indicator marked with **

29. Increased importance of monitoring - Delegated and Implementation Acts • Model for the OP (IA) • Performance framework: arrangements to ensure a consistent approach for determining milestones and targets and for assessing the attainment of the milestones and targets (IA) • Performance framework: financial corrections criteria (DA) • Data to be recorded and stored in computerised form (DA) • Model for the annual and final implementation report (IA) • Model for the progress report (IA)

30. 1. Model for the Operational Programme (IA) • Performance framework for the priority axis • Broken down by Fund • Broken down by category of region • Key implementation steps, financial indicators, output indicators and where appropriate result indicators • Indicators are set at IP level, but the indicators for the performance framework have to be aggregated Indicators and implementation steps must be representative for the priority axis • Milestones for 2018 and targets for 2023

31. 2. Performance framework (IA): arrangements to ensure a consistent approach for determining milestones and targets and for assessing their attainment Main elements of the Implementing Act: • Arrangements for documentation of the establishment of milestones and targets • Basic requirements for different types of indicators • Arrangements for determining milestones and targets • Arrangements for the verification of the attainment of milestones and targets

32. 2. Performance framework (IA): arrangements to ensure a consistent approach for determining milestones and targets and for assessing their attainment2.1. Arrangements for documentation of the establishment of milestones and targets • Methodologies and criteria toselect indicatorsfor PF ensuring milestones and targets comply with criteria of Annex II, § 3 CPR • Use of data/evidence and calculation method to estimate the value of milestones and targets • Rationale for the selection of output indicators, including explanation on share of the financial allocation represented by the operations which will produce the outputs + method to calculate the share -must represent more than 50% of financial allocation to the priority • Info on how methodology to ensure consistency in the PF has been applied in accordance with the provisions of the Partnership Agreement • Rational of selection of result indicators and key implementation steps

33. 2. Performance framework (IA): arrangements to ensure a consistent approach for determining milestones and targets and for assessing their attainment2.2. Requirements for indicators and key implementation steps • Milestones and targets to be set at level of the priority • In case of multi-fund/multi-category of regions priority axes: breakdown by Fund and by category of region • For financial indicators: M&T refer to total amount of eligible expenditure entered into accounting system of certifying authority… • For ESF output indicators M&T refer to achieved value for fully or partially implemented operations • Key implementation steps refer to an important stage in delivery of a priority, with verified completion, and expressed as number or percentage. • Result indicators used where appropriate, closely linked to interventions

34. 2. Performance framework (IA): arrangements to ensure a consistent approach for determining milestones and targets and for assessing their attainment2.4. Arrangements for verification of the attainment of milestones and targets • To be assessed at priority axis level • In case of multi-fund/multi-category of regions priority axes: assessment by Fund and by category of region • Achievement of milestone/target: all indicators of the performance framework of the priority axis have attained at least 85% of milestone (2018) or target (2023) value • Serious failure to achieve M&T: • Max 2 indicators/priority axis: any of the output or financial indicators failed to attain at least 65% of the milestone/target value • More than 2 indicators/priority: at least 2 of the output or financial indicators failed to attain at least 65% of the milestone/target value

35. 3. Performance framework (DA): financial corrections criteria Conditions for applying financial correctionsat end of programming period on basis of final implementation report: • Serious failure to achieve targets liked to financial indicators or output indicators • Max 2 indicators/priority: any of the indicators failed to attain at least 65% of target value • More than 2 indicators/priority: at least 2 of the indicators failed to attain at least 65% of target value • Serious failure is due to clearly identified implementation weakness • Commission previously communicated to MAthe clearly identified implementation weakness • MS failed to take necessary corrective action • No socio-economic or environmental factors, no significant changes in the economic or environmental conditions in a MS…seriously affecting implementation of priorities concerned

36. EC reporting (Art. 46bis) Reporting by the EC and debate on the ESI Funds Three types of report: summary report and strategic reports and annual progress reports Summary report: Summary report based on AIRs, including summary of evaluations Starting in 2016, to Council, EP, ECOSOC, CdR Strategic report: In 2017 and 2019, based on progress reports Annual Progress Reports: Starting in 2018, every two years To spring summit, ESI Funds contribution to Europe 2020 Regional Policy Cohesion Policy

38. Monitoring: Regulatory requirements on INDICATORS

39. List of common ESF and YEI indicators • Structure of Annex I and II • Annex I - common ESF indicators • (1) Common output indicators on participants • (2) Common output indicators on entities • (3) Common immediate result indicators • (4) Common longer-term result indicators • Annex II - Youth Employment Initiative indicators • (1) Immediate result indicators • (2) Longer-term result indicators

40. Annex I Common indicators

41. Output indicators on people Employmentstatus • unemployed, including long-term unemployed* • long-term unemployed* • inactive* • inactive, not in education or training* • employed, includingself-employed*

42. Output indicators on people Age • below 25 years* • above 54 years* • participants above 54 years who are unemployed, including long-term unemployed, or inactive not in education or training * Education • with primary (ISCED 1) or lower secondary education (ISCED 2)* • with upper secondary (ISCED 3) or post-secondary education (ISCED 4)* • with tertiary education (ISCED 5 to 8)*

43. Participants above 54 years who are unemployed, including long-term unemployed, or inactive not in education or training Above 54 years

44. Output indicators on people Disadvantaged participants • participants who live in jobless households* • participants who live in jobless households with dependent children* • participants who live in a single adult household with dependent children* • migrants, people with a foreign background, minorities (including marginalised communities such as the Roma)** • disabled** • other disadvantaged**

45. Participants who live in jobless households* All household members either unemployed or inactive Household – housekeeping/social unit: • having common arrangements; • sharing household expenses or daily needs; • in a shared common residence. -> one person living alone / group of people, not necessarily related -> living at the same address Excluded: - Households composed solely of students. - Collective / institutional households (hospitals, old people’s homes, residential homes, prisons, military barracks, religious institutions, boarding houses and workers’ hostels, etc.)

46. Participants who live in jobless households with dependent children * Subindicator of Participants who live in jobless households* Dependent children • all children under 17 years of age and • persons between 17-24 years of age who are economically dependent on their parents -> Participant can be any household member (parent, dependent child, other household member)

47. Participants who live in a single adult household with dependent children* NO subindicator to "jobless household", but same definitions regarding • household • dependent children

48. Output indicators on people Data to be submitted in June 2017 Disadvantaged participants • homeless or affected by housing exclusion* • from rural areas* The data on participants under the above two indicators are to be provided in the AIR as specified in Article 44(3) of Regulation (EU) No [...] CPR (i.e. in 2017). They are to be collected based on a representative sample of participants within each investment priority. Data for participants about rural areas are to becollectedat LAU 2 (local administrative unit, former NUTS 5).

49. Homeless or affected by housing exclusion National definition or ETHOS (European Typology of Homelessness and Housing Exclusion) definition – classification of four living circumstances : 1. Rooflessness (living rough / emergency accommodation), 2. Houselessness (in accommodation for the homeless, in women's shelters, in accommodation for immigrants, people due to be released from institutions and people receiving long-term support due to homelessness), 3. Insecure accommodation (insecure tenancies, under threat of eviction or violence), 4. Inadequate housing (unfit housing, non-conventional dwellings) -> Reference "Confronting Homelessness in the European Union"

50. From rural areas • Rural areas are to be understood as thinly populated areas according to the Degree of urbanisation (DEGURBA category 3) classification. • Thinly-populated areas means that more than 50 % of the population lives in rural grid cells. • The data shall be collected at the Local Administrative Unit level of LAU 2 (local administration/communes).The DE-GURBA category 3 shall be established according to http://ec.europa.eu/eurostat/ramon/miscellaneous/index.cfm?TargetUrl=DSP_DEGURBAheader "for reference year 2012".