Ex post evaluation of a sample of projects co-financed by the Cohesion Fund (1993-2002)

1. Ex post evaluation of a sample of projects co-financed by the Cohesion Fund (1993-2002) EEA Financial Mechanism, Brussels 13 June 2005

2. The Cohesion Fund • Established in 1993 • Aim: to assist least prosperous Member States (<90% EU average) • Total support: • 1993-1999: € 16 bn • 2000-2006: € 18 bn • 50% transport, 50% environment • A project approach instead of programme approach of ERDF • Level of assistance 80-85%

3. The evaluation • Required by regulation, however: • more than just accountability • learning experience just as important

4. The evaluation: 1993-2002 • Overall commitments under CF 1993-2002: 25 bln Euro • Nearly 1400 projects, of which 1060 in environment, 300 in transport • Average budget per project: • 11.5 mln Euro in environment • 40 mln Euro in transport • sample of 200 completed or almost completed projects, with focus on first period

5. Evaluation on 6 evaluation aspects • Appropriateness • Effectiveness • Efficiency • Impact • Management and implementation system • Community value added

6. Appropriateness – National needs • Three periods: • Interim facility: lack of project pipeline; hastily prepared projects • First period 1994-99: projects increasingly available; increasing fit with national strategies or needs assessment • Second period 2000-06: improved planning & programming of projects • Majority of projects fulfil national needs in general terms. But the way in which are usually not made explicit and the contribution of the projects are not always quantified

7. Appropriateness – Community Policy • The projects reviewed are generally in line with Community policies • Remarks: • Some projects do not have a clear relation with Community Policies (in the early period!) • Some (road) project are part of TEN, but their national impact is far more important • Shift towards high speed passenger rail neglects the rail freight sector, which deserves attention from sustainable transport point of view

8. Appropriateness – National procedures & criteria • In beginning “maturity” and “eligibility” of projects were most important criteria • Over time these criteria have remained important, but others have been added • Still projects have been selected that were not ready in terms of technical feasibility, or did not have highest priority • What is missing: problem descriptions, analysis of alternative options • CBA not always elaborate (or not carried out) • Long delays in approval are result (up to > 1 year, average 6-10 months)

9. Effectiveness – Outputs and Results • Generally no quantified objectives were set ex ante for results and goals • Final reports lack quantified information • In 50% of the reviewed projects outputs were fully realized (30% no data) • In 60/70% of the reviewed projects goals and results (mostly not quantified!) are deemed to have been or are expected to be realized

10. Effectiveness – Utilization, External factors • For many sample projects not sufficient information on utilization available • Of those for which information is available, 65% of projects are fully utilized • External factors usually not identified ex ante • Ex post there seems to be strong impact from external factors in 17% of the cases (32% no data), in 41% of the cases no or only small impact of external factors • Main ‘external’ factors identified (not all are external): • Public protest • Archeological factors / habitats • Weather conditions • Economic growth faster/slower • Land purchase

11. Efficiency – Time scale and costs • Only 20% of projects more or less stick to original time schedule, 30% show delay of more than 2 years • Reasons: • Insufficient preparation of projects/technical reasons • External factors (sometimes foreseeable) • Opposition from local population • Lack of management capability

12. Efficiency – Cost deviations • Cost overruns appear on more than 50% of projects • In 20% of projects cost overruns are less than 10% • In 25% of cases overruns are more than 20%. • Average cost overrun 15-20% • Generally time delays are more frequent than cost overruns (co-financing budget is constraining) • Main causes for cost & time overruns are: • Ill preparation of projects • External factors • Community oppostion • Lack of managerial capability

13. Efficiency – Co-financing rate • No clear criteria for setting co-financing rate (I.e. 80 or 85%), except for Spain: national level 85%, regional level 80% • Generally net revenues from projects have been taken into account (user charges), but mostly maximum level of co-financing used since fees are set/assumed at level of operating and maintenance costs • Full application of polluter pays principle is discouraged by co-financing mechanism • Ex post much more variation in co-financing due to cost overruns

14. Impact – Ex post CBA • Many difficulties in establishing ex post ERR (RERR) • Lack of data (e.g. on output) • Ex ante CBA weak or no existing (or only financial) • Ex ante CBA’s not always clear in methodology • RERR generally lower than ERR, but in most cases acceptable; typical range for transport projects: 10-25%; for environment projects 0-20% (avg 12%)

15. Impact – Economic impact • At project level only info available on temporary employment (direct and indirect), structural effect usually not known • At regional level impact can be substantial as seen from model runs LSE. Long run additional employment from larger projects can be up to 10,000 jobs (exceptional case: 16,500 jobs of Tagus crossing!) • Impact on regional growth differs considerably between regions. Usually effect less than 1% of the national or regional income.

16. Management and implementation • Management and implementation systems similar between countries: National managing authority with MA at sector level; More centralization over time in implementation bodies • Monitoring system is similar with monitoring committees and monitoring on the basis of physical and financial progress indicators • Administrative costs: difference between CF and SF does not seem to be significant. However, different timing of input, with more elaborate time needed at programming stage in SF, during implementation in CF

17. Community value added • CF played key role in improving transport infrastructure, drinking water supply, wastewater treatment, solid waste management in the CF4 • CF stimulated development of sector strategy and focus in sector policies • CF stimulated introduction of and improvement of techniques during project cycle management, from identification up to monitoring

18. Recommendations – Managing authorities and beneficiaries • Select only mature projects! • Adopt multi-annual planning approach • Create pipeline of projects • Request active public consultation • Request fully developed technical studies • Apply technical quality assurance • Offer methodological support to beneficiaries! CBA methodology, indicators

19. Recommendations – Managing authorities and beneficiaries (2) • Treat water supply, sewerage construction and waste water treatment in an integrated way! • Use masterplans • Request measurable and quantified goals, results and impacts! • Discuss projects with Commission before submission, check information needs!

20. Recommendations – Managing authorities and beneficiaries (3) • Ensure adequate, professional management of projects • Establishment of clear managerial body • Establish one managing body for groups of smaller municipalities • Give central assistance on administrative and financial matters • Central pre-funding system can greatly facilitate start of projects!

21. Recommendations – Commission • Additional (technical) quality assurance of projects • Offer overall methodological support • Clustering of projects only if really integrated • Address double “PPP” dilemma • Address change of ownership (public to private • Improved reporting requirements w.r.t. impacts (indicators!)