DIPARTIMENTO PER LE INFRASTRUTTURE, GLI AFFARI GENERALI ED IL PERSONALE

1. A comparative analysis of assessment approaches. Six cases from EuropePaolo Beria, Michele Giove, Fabio Croccolo, Nicolino Carrano, Andrea Salza, Rossella Napolitano

2. DIPARTIMENTO PER LE INFRASTRUTTURE, GLI AFFARI GENERALI ED IL PERSONALE DIREZIONE GENERALE PER LO SVILUPPO DEL TERRITORIO, LA PROGRAMMAZIONE ED I PROGETTI INTERNAZIONALI NUCLEO DI VALUTAZIONE E VERIFICA DEGLI INVESTIMENTI PUBBLICI This paper is the outcome of the first work package of a research programme conducted by Ernst&Young and TRASPOL on behalf of the Italian Ministry of Transport. The programme aims at the proposal of a new assessment methodology for Italy.

3. contents Introduction The country sample Comparisons and specificities Conclusions and directions for the future

4. Introduction Paper aims The paper aims at analysing and comparing the different approaches to Economic Assessment present in six EU countries • Italy, • France, • United Kingdom, • Germany, • the Netherlands, • Sweden • The same tool (CBA) must be “designed” differently according to the role is attributed to it (e.g. ranking many schemes vs. authorising a scheme) Methodological comparison Procedural comparison How CBA is structured How CBA is used (HEATCO, EVA TREN, etc.)  This paper

5. Approach used by public decision maker Who assesses? Participation Introduction Structure of the comparison A simplified scheme has been introduced. Countries analysed have been firstly described and then “fitted” to the scheme. Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Public dec. maker Public dec. maker Public dec. maker Proponent Proponent Indep. evaluator Indep. evaluator Indep. evaluator Indep. evaluator yes No Yes No Yes No

6. contents Introduction The country sample Comparisons and specificities Conclusions and directions for the future

7. Approach used by public decision maker Who assesses? Participation The country sample UK Assessment in UK permeates deeply the decision process of every transport scheme. (strategy)  problem  proponent  alternatives + debate  best alternative Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Proponent Indep. evaluator Indep. evaluator yes No Yes No Yes No

8. The country sample UK • British system is among the most advanced of the sample. CBA pervasively used in every decision; • Periodically updated and extended. The method is widely accepted; • Methodologically conservative: only “sure” and shared upgrades are included (e.g. wider benefits). Schemes must justify essentially with direct transport benefits: indirect benefits may help, but cannot justify a project alone. • The process tends to be “project-focused” rather than centrally planned. “Liberal” viewpoint: the State fixes the rules, but does not (heavily) plan. This can also be seen as a limit; • Transparency and clarity in the outcomes (AST); • Public involvement exists but reveals very costly (£ and uncertainty).

9. Approach used by public decision maker Who assesses? Participation The country sample The Netherlands Assessment & transport planning are part of the broader activity of land use planning. economics land use planning  environmental protection Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Public dec. maker Public dec. maker Proponent Proponent Indep. evaluator Indep. evaluator Indep. evaluator yes No Yes No Yes No

10. The country sample The Netherlands • Dutch system is clearly CBA based, but inside the overall framework of land use planning. MCA excluded because not sufficiently transparent. Detailed guidelines are available. Approach very rich; • CBA is used both • to include/exclude a project at the planning level (guidelines compulsory), • In the feasibility study to support design (guidelines not compulsory); • CBA result is not binding (yes/no), but different decisions (e.g. for land use reasons) must be formally justified  yes freedom, no arbitrariness; • Methodology includes wider benefits, distributive analysis and risks; • Participative moments exist along the whole project cycle; • CBA positively influenced decisions (downsizing of pjs, postponement of bad pjs).

11. Approach used by public decision maker Who assesses? Participation The country sample Germany Assessment in Germany is a centrally managed function, used essentially to rank concurrent projects. Federal structure. Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Public dec. maker Single states Proponent Indep. evaluator Indep. evaluator yes No Yes No Yes No

12. The country sample Germany • The Federal Infr. Tr. Plan is the pivot of the system and includes the CBA evaluation of all projects submitted by States and Agencies to rank them; • After inclusion in the FTIP the “good” projects (“yes/no”) are devolved to States that manages the further design&assessment phases (“how”); • The criterion is a benefit:cost ratio (N/K). No arbitrariness allowed, in theory; • Actually, local stakeholders consultation, budget allocation and other criteria (environment & risks) distort the absolute results of CBA; • Common (country-wide) datasets and simulation models  comparativeness; • Methodology have some problems (double counting).

13. Approach used by public decision maker Who assesses? Participation The country sample Sweden From a broad list of projects, CBA selects and ranks. From three short lists (initial plan, base, extended), a plan is produced. Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Public dec. maker Public dec. maker Proponent Proponent Indep. evaluator Indep. evaluator Indep. evaluator yes No Yes No Yes No

14. The country sample Sweden • One of the most advanced systems. Actually incentivised good projects; • Independent agencies manage planning & evaluation trough standardised CBA; • CBA is not the sole criterion, but dominates: welfare economics + environment and safety + regional development. Moreover, budget of single modes is pre-defined (bias); • CBA is not binding (see the Netherlands), but its result is taken seriously into consideration: a welfare-destroying scheme can hardly be selected; • standardised modal guidelines, harmonisation ongoing. Same methodology from ranking to detailed design. Standard input values or methodologies to calculate. National simulation model; • Ex-post analysis compulsory; • Substantial participative moments;

15. Approach used by public decision maker Who assesses? Participation The country sample Italy Italy moved from a centrally planned approach to a project-focused approach: evaluation takes place only at the project level. Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Public dec. maker Public dec. maker Proponent Indep. evaluator Indep. evaluator yes No Yes No Yes No

16. The country sample Italy • Mandatory evaluation, carried by proponents; • Only generic guidelines are present. Some standard values given (VOT, external costs), but no methodological details (e.g. surplus calculus) or national simulation models; • CIPE Committee and Regions decide (yes/no) on the basis of feasibility project; • Some priority projects decided by government before a CBA is carried. Some degree of arbitrariness is also left at the project level; • Comparative character of evaluation is not sufficiently stressed.

17. Approach used by public decision maker Who assesses? Participation The country sample France A double system is presently in use in France: CBA vs. MCA. Historically, CBA was the basis of the decision process. Strategy Prioritisation of a list of schemes Single scheme Ex-post Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) No ex-post (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Required but not systematic (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++) Ex-post compulsory (++) BUILDING PHASE Public dec. maker Public dec. maker Public dec. maker Proponent Proponent Indep. evaluator Indep. evaluator Indep. evaluator yes No Yes No Yes No

18. The country sample France • The relationship btw. MCA and CBA is not clear, no one prevails; • Detailed CBA is used at the project level. The previous planning level is less formalised. However, CBA is used very early in the project cycle, when many alternatives are still at stake; • Guidelines are complete; • Participative procedures deeply shape the process. CBA is explicitly used in such debates. Debates’ outcomes are not binding (incl. CBA), but transparency is stressed.

19. contents Introduction The country sample Comparisons and specificities Conclusions and directions for the future

20. Comparisons and specificities Two models We recognise two “models”, one focusing on single schemes assessment (“bottom-up”), the other using assessment to sharpen choices from plans (many projects) to one project and to single alternatives (“top-down”). Problem Project Alternatives Project Alternatives Plan

21. Approach used by public decision maker Comparisons and specificities Bottom-up models We recognise two “models”, one focusing on single schemes assessment (“bottom-up”), the other using assessment to sharpen choices from plans (many projects) to one project and to single alternatives (“top-down”). Bottom-up model Strategy Prioritisation of a list of schemes Single scheme Validation of assessment UK IT No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++)

22. Approach used by public decision maker Comparisons and specificities Top-down models We recognise two “models”, one focusing on single schemes assessment (“bottom-up”), the other using assessment to sharpen choices from plans (many projects) to one project and to single alternatives (“top-down”). France is in between… Top-down model Strategy Prioritisation of a list of schemes Single scheme Validation of assessment No public planning (-) No prioritisation /ranking (-) No compulsory guidelines (-) No validation (-) FR UK SE and NL DE Definition of strategies only (+) Informal but explicit ranking (+) General guidelines (+) Validation / Certification (+) Planning (++) Formal economic assessment (++) Complete economic assessment (++) Peer Review (++)

23. contents Introduction The country sample Comparisons and specificities Conclusions and directions for the future

24. Conclusions and directions for the future Issues for debate Each model has different “needs” in terms of how assessment must be carried, also including technicalities. BOTTOM-UP models Must provide very detailed and compulsory guidelines to proponents in order to avoid free riding and opportunistic behaviours. Should consider the introduction of peer-reviews to validate proponents’ evaluations. Do not necessarily need centralised structures to compare among projects. Need rules to manage budget constraints. Should not leave politicians any freedom to finance projects not selected by CBA. Their role must limit to rules and goals (possibly translated into shadow prices).

25. Conclusions and directions for the future Issues for debate Each model has different “needs” in terms of how assessment must be carried, also including technicalities. TOP-DOWN models Must stress comparativeness and transparency of the process. Can use common inputs (e.g. value of time) to translate political directions into choices (e.g. higher external costs to promote environmental friendly transport). Can even avoid precise assessment at the project level. Need a strong and informed central structurein the Ministry to manage guidelines, data, models and results. Can leave politicians some freedom to finance also projects not included in the ranking, if transparency is guaranteed.

26. Thank you for your attention! paolo.beria@polimi.it