Evaluation SG Seminar 9 March 2009 Svend Jakobsen and Sara Hector, SG.C3

1. Evaluation SG Seminar 9 March 2009Svend Jakobsen andSara Hector, SG.C3

2. Main topics • What is the focus of the Commission’s evaluations? • How does evaluation fit into the Commission’s ABM/SPP cycle? • The link to Impact Assessment and Better Regulation • Who are involved in evaluating Commission’s activities? • How are the Commission’s evaluations carried out and used?

3. What is the focus of the Commission’s evaluations? Evaluation is… “an evidence-based assessment of actual or expected results and impact” or “a judgment of interventions according to their results, impacts and needs they aim to satisfy”

4. What is the focus of the Commission’s evaluations? • Main purposes of evaluation • To provide input for setting political priorities and contribute to the design of interventions • To assist in an efficient allocation of resources • To improve the quality of interventions • To report on the achievements of interventions (i.e. accountability)

5. What is the focus of the Commission’s evaluations? • What do we evaluate? • Expenditure programmes • Regulatory activities • Communication and networking • Internal policies and services • Agencies • Pilot projects and preparatory actions • At different points in time • Forward looking • Backward looking

6. What is the focus of the Commission’s evaluations?

7. What is the focus of the Commission’s evaluations? Example of an evaluation question from AIDCO Question:To what extent has EC support improved the capacity of the educational system to enrol pupils from disadvantaged groups? Criteria: Capacity of the primary school system to enrol pupils from ethnic minority X with satisfactory quality Capacity of the primary school system to enrol pupils from the poorest urban areas with satisfactory quality Indicator: Number of qualified and experienced teachers per 1000 children of primary-school age in areas where ethnic minority X concentrates Number of enrolled pupils of primary-school age from ethnic minority X

8. How does evaluation fit into the Commission’s ABM/SPP cycle?

9. The link to Impact Assessment and Better Regulation • IA and ex-ante have different origin and legal base • IA and ex-ante share the same purpose • IA and ex-ante have similar requirements and coverage and are often merged • An increasing need for retrospective evaluations of key regulatory measures • Evaluation results should feed into the IA

10. Who are involved in evaluating Commission’s activities? • Main actors: Member States and the Commission • In the Commission: DGs evaluate their activities, SG provides central support and coordination and a network shares best practice • In the DGs: evaluation functions in cooperation with operational units

11. How are the Commission’s evaluations carried out and used? • DG EAC's interim evaluation of the Erasmus Mundus programme in 2007 • Aims: address relevance, utility, efficiency, effectiveness, sustainability and added value • Method: desk research, surveys, interviews and case studies • Main use: Improve implementation of ongoing programme and input to the next programme

12. How are the Commission’s evaluations carried out and used? • Evaluation of legislation: The Directive on Lifts (95/16/EC) • Aims: provide information on application of the Directive for the re-examination required by Art. 16 • Method: quantitative data collection, site visits and qualitative assessments • Main use: improve implementation through input to a guide of application

13. Evaluation For further information: http://ec.europa.eu/budget/sound_fin_mgt/evaluation_en.htm http://www.cc.cec/budg/man/eval/eval_en.html