STRUCTURE OF PRESENTATION

1. SPP BUILDING IMPLEMENTATION CAPACITYTraining seminar on evaluation Prague 26-27 February 2004 A VIEW FROM THE COMMISSION ON EVALUATION IN THE CZECH REPUBLICPascal BoijmansEuropean CommissionDG Regional Policy 1

2. STRUCTURE OF PRESENTATION • Experience with ex-ante evaluation in the Czech Republic • strong points • points that need further attention • Building up evaluation capacity in the Czech Republic • evaluation plan • organisation • financing • Evaluation vs. Monitoring • Conclusions and recommendations • Sources of information 2

3. EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC • NDP: Berman Group, Charles University, Faculty of Science • OP Industry & Enterprise: VSE, March Consulting, IFO Institute • OP HRD: National Observatory for Employment and Training • OP RDMFA: Institute for Economic and Environmental Policy, University of Economics (VSE) • OP Environment: Cityplan • OP Transport: DHV CR • OP Tourism: Europroject, Redeco • JROP: VSE • Prague Obj. 2: DHV CR • Prague: Obj. 3: National Observatory for Employment and Training 3

4. EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC • genuine efforts to follow EU guidelines: working paper no 2 on ex ante evaluation • lessons from the past • socio-economic analysis • choice of strategy and priorities: internal and external coherence • quantification of objectives • expected socio-economic impact and allocation of resources • implementation arrangements • separate Strategic Environmental Assessments (SEAs) for each programme • involvement universities and research institutes 4

5. EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC • standard Terms of Reference • restricted tender or direct agreement • guarantee of independence evaluator • internal or external evaluator? • financing • ex ante evaluations 2004-06: national budget • interactive process with programming exercise 5

6. EXPERIENCE EX-ANTE EVALUATION IN THE CZECH REPUBLIC • Case study: ex-ante evaluation NDP by Berman Group • consortium of Czech and foreign experts • involvement Dept. of Social Geography and Regional Development, Faculty of Science, Charles University • interactive exercise with programming: large influence on final NDP • concise report with clear recommendations • input for negotiation mandate EC 6

7. BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC • COMMITMENT IN CHAPTER 21 FOR THE CZECH REPUBLIC: • Binding rules will be set up for the evaluation of programmes and projects • a permanent evaluation body will be created within each Managing Authority ate the level of OPs • MRD will be responsible for the overall coordination of separate evaluation units 7

8. BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC • TEXT IN COMMUNITY SUPPORT FRAMEWORK STATES: • A part of the budget under heading TA will be reserved to develop the evaluation capacity for 2004-2006 programmes and for future 2007-2013 programmes. • Examples of areas to be covered are macro-economic assessment, reliability of indicator system and horizontal priorities • A specific evaluation unit in Ministry for Regional Development is in charge of coordination 8

9. BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC • DEVELOPMENT OF AN EVALUATION PLAN • Evaluation is a learning instrument • Differentiation in approach between CSF and OPs • CSF: • horizontal issues and priorities • Lisbon strategy • macro-economic assessment (HERMIN model) • reliability monitoring and indicator system • preparation for period 2007-2013 • OPs: • ex-ante next period 2007-2013 • sectoral issues • Coordination role for MRD: exchange of information • Focus on limited number of evaluations 9

10. BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC • ORGANISATIONAL ASPECTS • Specific evaluation unit in MRD • Interministerial working group • Strengthen evaluation capacity within national authorities • Development of tools how to use and spread added value of evaluations • Discussion of outcome evaluations in Monitoring Committees: trigger for in depth discussions 10

11. BUILDING UP OF EVALUATION CAPACITY IN THE CZECH REPUBLIC • FINANCING • Technical Assistance: rule no. 11 of Reg. 1145/2003 (not capped) • separate heading in TA-plan • Eligible expenditure: • external expertise • additional staffing (capped) • training • workshops, seminars, study visits 11

12. EVALUATION VS. MONITORING • different activities, but clearly depending on each-other • importance of well functioning and reliable monitoring system is a precondition for good evaluation • MSSF: Central, Benefit, Monit • VIOLA • and other systems • training of staff on input of data • limited number of high quality and representative indicators • baseline • target • source of information • Guidelines EC on indicators for new Member States 12

13. CONCLUSIONS & RECOMMENDATIONSPart I • Development of standard Terms of Reference for all evaluations • EC: preference for restricted tenders • Ex-ante evaluation: parallel and interactive process with programming exercise • Continue involvement of universities and research institutes • Evaluation reports: concise and clear recommendations 13

14. CONCLUSIONS & RECOMMENDATIONSPart II • Evaluation plan: focus on limited number of evaluations • Differentiation in evaluation approach between CSF and OPs • Develop tools how to use best the added value and results of evaluations • Develop evaluation expertise within national authorities • Assure reliability of monitoring systems as basis for evaluation tasks 14

15. INFORMATION SOURCES • Evaluation Section of Inforegio website: • http://europa.eu.int/comm/regional_policy/sources/docgener/evaluation/evaluation_en.htm • Commission working documents • Evaluations undertaken for the Commission • The new Evaluation GUIDE (forthcoming) 15