Evaluation Arrangements for 2007-2013: A Decentralized Approach

1. Evaluation Arrangements for 2007-2013:A Decentralized Approach Laura Tagle Evaluation Unit – Department for Development Policies – Ministry for Development Rossella Tarantino Evaluation Unit – Basilicata Region Bruxelles 26-27 June 2008

2. Presentation Outline • Presenters’ background: collective evaluation capacity building • Overall approach to evaluation: decentralization, responsibility and Evaluation plans • Main issues: • Evaluation questions • Involvement of social and economic partners • A regional perspective on a “central” endeavour

3. National Evaluation System • Work group: collective work for evaluation capacity development • Individuals from: • Evaluation Unit of Ministry for Development (responsible for work group coordination) • ISFOL (Agency responsible for evaluation of ESF) • INEA (Agency responsible for evaluation of Rural Development) • Regional Evaluation Units

4. Decentralization and responsibility • Decentralization: • each Regional government responsible for evaluating its interventions • Coordination ministry responsible for evaluating from “central” point of view • Regional ownership: each Region asks the evaluations it needs and times it for when it needs the results • No common evaluation questions • No prescribed timing • Responsibility: • No sanction • Evaluation plans and evaluations are public • National Evaluation System guarantees guidance, training, support, systematic observation and reputational mechanisms

5. Evaluate the effects of regional policy • Until the end of 2009, mainly ex post evaluations of past interventions • Evaluate EU- and nationally funded interventions • Evaluate effects of joint interventions on areas/groups • Evaluation questions on specific issues, groups, areas, NOT generic “program”

6. Steering Groups • Ensure involvement of socio-economic partners: Steering Group at Plan level • Ensuring qualityand independence of each evaluation: Steering Group (+ other methods) at each evaluation level. No MA. Risks: • creating a new profession: “Steering Group member” • Duplicating the problems of Monitoring Committees

7. Evaluation plans • One plan per Region/central ministry regarding all programs (past and present) (unitary approach) • Need to build a new internal organization • 30 programs • Only two Regions are still developing the plan (out of 21)

8. Evaluations planned for 2008 e 2009 • Research & innovation : 6 • Enterprises: 3 • Environment: 3 • Cultural heritage: 1 • Gender: 3 • Area-based: 9 • ICT: 2 • Human Resources: 5 • Education: 1 • Urban policies: 1 • Transport: 2 • Implementation analysis: 13 • Program evaluation: 18

9. Satisfaction at innovation spread, but • Out of 67 evaluations planned for 2008-2009, 31 (+ all the regional development evaluations) still have a generic “Structural Fund program” focus: • Inertia in accepting innovations from regulations and from NSF • Need for general overview • Incomplete or contradictory support from EC • SNV: • strengthening our common work on ECD with EC • Increase capacity of national coordination agencies • Further work needs to be done to involve social & economic partners • Is a Plan Steering Group the right way to go about it?

10. The National Evaluation System and the evaluation plans • “Guidelines for the organisation of the evaluation activities of the regional policies: the evaluation plans for 2007-13”; • Working groups that went through the evaluation plans drafted by Regions and Ministers according to a peer group approach. These groups were created on volunteer basis and composed of both regional and national representatives. This cross-regional and national scrutiny of the evaluation plans enabled a pooling of experiences and know-how. for instance, after going through the evaluation plan of the Liguria Region I changed/improved the evalution plan of my Region

11. The evaluation plan: the unitary approach • Each Region has a unitary strategy implemented through OP’s • One Evaluation Plan for Each Region that assess: • ERDF OP • ESF OP • EARDF OP • OP funded by National Fund for deprived areas • Interventions financed by regional resources

12. The evaluation plan: the unitary approach to assess all the development policies implemented by the Region –independently by the financial source – in order to have a global overview of the effects that these policies produce. Obviously in the respect of: • Specific rules concerning each Fund; • The need of the European Commission and of the single Managing Authorities to underline the specific community added value.

13. The unitary approach- why? • To coordinate the evaluations and avoid overlapping and duplication; • To build a global overview of the individual evaluations and thus have a big picture of the effects of the regional development strategy; • To assess the results produced jointly by different actions on the same area or target group • To have a shared knowlegde to improve the quality of programs and of their implementation

14. The unitary approach-how Specific evaluation activities focus on themes/areas/target groups and take into account results produced by different interventions An ex-post evaluation in my region concerns the information society and the contribution to: • The quality of the public services (i.e; health) • The creation of community networks (i.e. Schools) • The territorial and social inclusion • The competitive capacity of SMEs How to deal with organizational and financial issues? • Pooling of funds (liguria) • Steering Group with the participation of all Managing Authorities and departments involved in these policies (Basilicata) • To address specific national resources (Emilia oRmagna)

15. The unitary approach: not easy • In some regional plans there is a divide between the ERDF and ESF evaluations • In the implementation phase, the unitary approach is even more difficult Therefore, for us (Regions) it is important: • EC common approach • EC Common guidance • EC (DG Regio and DG Employment + DG Agri) joint meetings and analysis with National Evaluation Systems

16. EC support and guidance • The joint analysis on the mid-term evaluation conducted on 2004 was very useful and gave important hints to improve the quality and awareness of evaluations: • focus on specific questions, stakeholders involvement in the definition of questions and in the dissemination of evaluations results, emphasis on effects, etc • Guidance • Participation in our (SNV) ECD activities