Evaluation plans for 2014 -2 020 programming period in Poland Experience and new arrangements

1. Evaluation plans for 2014-2020 programming period in Poland Experience and new arrangements Ministryof Infrastructure and Development, Poland Athens, 15th May, 2014 Joint Evaluation Network Meeting

2. Institutional context for evaluation planning 7 persons 20 persons 60 persons 35 persons 43 persons Closer coordination as a main challenge for 2014-2010 period

3. Polish experience on evaluation planning 2007-2013 • 2007-2013 period: evaluation plans establishedfor NSRF and all operational programmes • Strategic evaluation plans for the whole 2007-2013 period • strategic goals for evaluation process • organisational arrangements (thematic steering groups) • main thematic areas • main evaluation capacity building activities • Annual evaluation plans • list of evaluation studies + description • evaluation capacity building activities

4. Polish experiences on evaluation planning 2007-2013 • All plans examined by the National Evaluation Unit • Shortcomings: • difficult to coordinate • no strategic integration of an evaluation process with implementation and intervention logic → annually planned evaluations mainly for current informational needs • no coherent evaluation agenda for the whole Cohesion Policy • Advantages: • high degree of flexibility

5. Evaluation planning 2014-2020 in Poland • Process of evaluation planning in Poland to be regulated by the Guidelines on evaluation activities in the 2014-2020 period (draft) • In the field of the evaluation plans the Guidelines contains: • general rules • obligations for institutions, and role of the main stakeholders • structure and content of evaluation plans • deadlines and logistic/organisational matters • list of obligatory evaluation studies (e.g. ex-ante, mid-term, ex-post, each priority evaluation, annual systematic review of evaluation studies, gender equality evaluation)

6. Evaluation planning – general principles • Evaluation process closely related to the intervention logic • evaluation plans (esp. scope and timing) reflect the intervention logic of the programme • intervention logic as a starting point for identification of the informational needs (what kind of and when evaluation evidence will be needed) • all planned evaluation activities should be justified in this context (2) Planning process closely coordinated by NEU in order to ensure the coherence of the evaluation system • twofoldaim of the planning: 1) effective evaluation of each programme 2) effective evaluation of the whole Partnership Agreement intervention • coherent evaluation agenda is needed → operational programme evaluations + evaluations in the PA level (meta-evaluations, systematic reviews, strategic and horizontal evaluations)

7. Evaluation planning – general principles • (3) Monitoring of evaluation plans implementation • whole evaluation process monitored by National Evaluation Unit • special emphasis on recommendation implementation process → Recommendations Implementation System (RIS) • annual reports on the progress of implementation of evaluation plans (esp. recommendations implementation)

8. Evaluation planning in Poland – proposed rules and procedures • Evaluation plans prepared for every operational programme by the Evaluation Units for the whole programming period • National Evaluation Unit preparing the Evaluation Plan for Partnership Agreement 2014-2020 • Annual update as a tool for changes and supplementation • Dilemmas: • Evaluation plan for the whole period contains all evaluation studies planned for each programme vs. contains only strategic and most important studies • Annual update of one plan vs. annual plans as a supplement for the strategic plan • Evaluation Units are obligated to ensure: • linkage between the planned evaluation studies and the intervention logic of the programme • high quality of evaluation studies • identification of needed data • resources needed for evaluation process

9. Evaluation planning in Poland – proposed rules and procedures • All evaluation plans and their annual updates accepted by National Evaluation Unit • NEU’s opinion based on the criteria of: adequacy, coherence, feasibility, and efficiency in order to ensure the coherence of the whole cohesion policy evaluation process • After the NEU decision, evaluation plans are accepted by Monitoring Committees

10. Main challenges for evaluation planning • strategic planning vs. current evaluation needs → annual update or annual plans as a tool for flexibility • effective integration of evaluation process with the intervention logic (and then implementation process) within evaluation plans → high expert and evaluation potential needed → cooperation with external experts + high quality trainings • effective coordination in order to establish coherent evaluation agenda for the whole Cohesion Policy→ National Evaluation Unit responsibility • identification and access to the needed data (in particular for counterfactual studies)

11. Preparation to evaluation planning in Poland • National Evaluation Unit prepared detailed Guidelines on evaluation activities in the 2014-2020 period with section dedicated to evaluation planning. • After the discussion with the OP Evaluation Units, NEU decided to support evaluation units, and to organise the evaluation planning process in closely coordinated manner. • NEU is going to launch the project: Process of preparing evaluation plans for Cohesion Policy 2014-2020. • In cooperation with experts on evaluation and sectoral matters • Main goals of the project: • Development of the evaluation planning methodology (taking into account intervention logic) • Preparation of evaluation plans for Operational Programmes and Partnership Agreement

12. Preparation to evaluation planning in Poland • Main stages of the project: • Pilot project – preparation of an evaluation plan for one chosen Polish region (Regional Operational Programme) • intervention logic analysis → expert panel • workshops with main Regional Operational Programme stakeholders and partners • preparation of the Regional Operational Programme Evaluation Plan • Preparation of the Partnership Agreement Evaluation Plan • presentation and discussion on a pilot project findings • intervention logic analysis → expert panel • workshops with main stakeholders and partners • preparation of the Partnership Agreement Evaluation Plan

13. Preparation to evaluation planning in Poland • Preparation of evaluation plans for all OperationalProgrammes • with the support of the NEU (help desk) and external evaluation experts • NEU opinion • Monitoring Committee approval • Sending plans to European Commission • Main products: • evaluation plans for OP’s and PA • methodology document on evaluation planning

14. EVALUATION PLAN OF KNOWLEDGE – EDUCATION – DEVELOPMENT OPERATIONAL PROGRAMME – MAIN ASSUMPTIONS

15. A ROLE OF EVALUATION PLAN • Programme evaluation management • Predictable information needs fullfilment • Evaluation WWW - (What When Who) • Rising awarness about preparations to be done • Transparency • Stakeholder’s interest taken into account

16. MAIN PRINCIPLES OF THE EVALUATION PLAN • 1 EP for 1 OP • Evaluation experts functionally independent • Evaluation capacity building • Social partners involvement • Long-term orientation = flexibility of provisions

17. FLEXIBILITY OF THE EVALUATION PLAN • EP should be a strategic oriented document. • List of evaluation studies is open and indicative. • The list covers only evaluation studies that are predictable. • Detailed information about the studies will be updated yearly.

18. ELEMENTS OF THE EVALUATION PLAN • Definition – Goals – Challanges and Principles for evaluation process • Data access and provisions • Process of coordination • Capacity building and evaluation culture • Use and dissemination of findings • Indicative list of planned evaluation studies

19. EVALUATION STUDIES CHARACTERISTICS • Subject /area • Purpouse and scope • Who • Evaluation approach planned • Tentative term – usually approximately • Budget – broadly estimated • Other information if necessary

20. TEMPLATE - EXAMPLE

21. TYPES OF EVALUATION STUDIES • Obligatory studies • Metaevalution, syntheses • Diagnoses, needs and barriers identification • Horizontal principles • Specific target groups oriented • Comparability studies • Capturing the effects • Ad hoc studies

22. MAIN SOURCES OF IDEAS • Research obligatory or necessary • Proposals from ex-ante evaluation • Proposals made by the IB’s • 2004-2006 and 2007-2013 experience • Policy makers needs – if possible to identify

23. Thank you for your attention! Tomasz.Kot@mir.gov.pl Michał.Sułkowski@mir.gov.pl Ministry of Infrastructure & Development POLAND