Assessment of Sustainability Indicators

1. Assessment of Sustainability Indicators Tomas Hak Charles University Environment Center, Prague Workshop on Ecoinformatics, EEA, 21-22 June

2. Project organization ASI speaks to the SCOPE's mandate to: • analyse, synthesize and assess existing knowledge rather than to undertake new research ASI project was implemented jointly by SCOPE and UNEP • together with the International Human Dimensions Programme on Global Environmental Change (IHDP) • and the European Environment Agency (EEA), under thesponsorship of the international Council for Science (ICSU) Workshop on Ecoinformatics, EEA, 21-22 June 2007

3. Project rationale (I) • almost 10 years passed since implementation of CSD Workprogramme on Indicators (1995) • first session of the UN CSD-1 in New York (1993) and CSD-2 (1994) • …….. • important milestone - SCOPE Volume 58 “Sustainability Indicators”, was published and distributed to all delegations at the UNGASS (UN General Assembly Special Session) in 1997. • nowadays, number of organisations (of all kinds) develop and use SDIs of several formats: • Comprehensive sets for all pillars of SD (e.g. UNCSD, OECD, …) • Headline indicators (governments – e.g. UK, cities – Common European Indicators, …..) • Composite indicators and indices (e.g. HDI of UNDP, ESI by Yale and Columbia University, Living Planet Index of WWF, Ecological Footprint, ……) Workshop on Ecoinformatics, EEA, 21-22 June 2007

4. Project rationale (II) • SDIs are used ever more extensively and intensively bya wide range users and in many different contexts • However, it does not necessarily follow that they are scientifically sound and/or used appropriately • The scientific community now faces a task: to analyse and evaluate the range of indicators presently used by a great number of institutions and decision makers at all levels

5. Growth in the number of indices R.Bandura: Measuring Country Performance and State Behaviour. A Survey of Composite Indices

6. Project goal • to generateasurvey of existing indicators • to provide a preliminary step for the development of a science-based assessment of existing SDIs (based on „ASI criteria“ • to show selected examples of the development and use of SDIs in different regions

7. Project organization (II) • Planning meeting (Steering Committee) • Background papers • Workshop (45 participants, summer 2004, Prague) • Publication - Overview and Cross-cutt Chapters, selected Background Chapters • Editing, finalizing the manuscript • Review process (internal, external) • Publishing • Distribution

8. ASI Workshop – WG I (Conceptual challenges) • Sustainability in general • Linkages between pillars of sustainability • Cultural differences • Democracy • Equity • North-South issues • Capacity building • Different temporal and spatial scales • Strong-weak sustainability • Vulnerability • Resilience • Adaptation • Carrying capacity • Irreversibility

9. ASI Workshop – WG II (Methodological frontiers) • Definitions, units • Legitimacy, status • History of development and use • Stability of measurements over time • Data demands, cost estimations, feasibility • Soundness of calculation methods • Used frameworks • Normativeness vs. objectivity • For aggregated indicators: aggregation methods, weighing • Unambiguous interpretation • Transparency • Robustness of results • Uncertainty issues • Limitations • Identification of gaps

10. ASI Workshop – WG III (Policy relevance ) • Scale: International, national, other (sectoral etc.) • Use in policy cycle • Stakeholders, users • Public participation • Usefulness – limits of applicability • Relation to policy targets, norms, limits (international, e.g. conventions), national, others • Naming and faming (shaming)

11. SCOPE Volume 67: „Sustainability Indicators: A Scientific Assessment” • edited by B. Moldan, T. Hak and A. Dahl • it focuses primarily on assessment of existing indicators (in addition, some new approaches are proposed) • 3 cross-cut chapters look in depth at conceptual, methodological and policy aspects of SDIs • the successes are analyzed – in 20 Chapters covering both methodological aspects and case studies (national, regional) • Chapters cover a wide geographical scope (Europe, America, Asia, • the Volume can be read at different levels (it is not written at the highest scientific level): it is aimed not only at researchers, policy planners, and decision makers, but also at the informed public interested in using indicators

12. SCOPE Volume 67: Structure 5 Parts (23 Chapters): • Cross-cutting issues • General approaches • Methodological aspects • System and sectoral approaches • Case studies

13. Main findings • useful progress since the Rio Earth Summit in 1992 adopted Agenda 21 and launched an international indicators process • many indicator sets have been assembled, countries have started their own indicator programs at the national level as called for by the UN Commission on Sustainable Development • many aspects of sustainability have been given a more precise definition or measure through indicators • methodologies are gradually becoming standardized, and policy decisions increasingly provide clear directions and targets • there is no ideal indicator that is fully encompasses all desired qualities – trade offs should be transparent and constraints limited

14. Impartial assessment ? • The project laid questions of a strategic character: • how to obtain significant results (to get all major indicator developers involved etc.) and not to offend any indicator provider? • There are basically 2 approaches: • work in a „compromise style“ • prepare a methodology for assessment based on objective criteria

15. Future development • At the moment, there is no international strategy or clear future direction • Also, there is no mechanism for international leadership in this area „anarchy“ continues some interventions and until survival of the fittestguiding the process

16. http://www.indi-link.net/

17. Background • The Commission – WG on SDIs – has developed a set of SDIs since 2001 • Both the framework and number of SDIs has evolved from several hundreds of SDIs to the current format • 155 indicators (of which 34 are not yet feasible and 11 others are replaced by proxies) • SDIs are structured in a „hierarchical theme framework“ (six priority areas of the 2001 strategy together with themes included in Lisbon strategy and those derived from the WSSD plan of implementation)

18. INDI-LINK Structure 1. Further development and improvement of the EU SD indicators (WP 1) 2. Assessment of interlinkages between different priorities of the EU SDS (WP 2) 3. Elaboration of conclusions for future SD policy making (WP 3) 4. Dissemination (WP4)

19. WP1 – Further development of SDIs • WP 1 is to further develop EU SDIs - review the state of the art in SDIs development on European, national and international levels - suggestwhich indicators should receive top priority in further methodological and data development - consider recommendations by the SDI Working Group with a particular focus on the development of „indicators to be developed“

20. Criteria for evaluation of the EEA Indicators Core Set • Policy relevance • Progress toward the targets • Available and routinely collected data • Time and space and temporal coverage • Timeliness • National scale and representatives • Clearness (and unambiguity) of indicators • Methodological foundation • Transparency and quality • Topics covered and priorities

21. Summary of environmental indicator selection criteria (US EPA) • Scientific validity (technical considerations) • Measurable/quantitative • Sensitivity • Reproducible • Representative • Reference value • Comparability • Practical considerations • Cost/cost effective • Level of difficulty • Programmatic considerations • Relevance • Program coverage • Understandable

22. SDIs Assessment criteria • Relevance • Policy relevance • Meaningfulness • Clear interpretation • Credibility • Scientific soundness • Data quality • Clear in content • Technical considerations • Data availability • Spatial and temporal scale • Comparability across countries

23. Use of SDIs Assessment Criteria (1) • Operationalization of the criteria through the measurable characteristics (markers) to allow consistent application (ideally – any assessor would result in the same score and rank of any indicator) • Assignment of points (grades) to each marker based on a 5-point scale in accordance with the defined ranking • Points are then totaled (summation of all markers, which are equally weighted) to provide a rationale for the overall ranking • Disaggregated results show strengths and weaknesses of the indicator in particular domain (relevance, credibility and feasibility) • The disaggregated picture is also provided “at glance” by a radial chart

24. Indicator Cluster criteria - grades Relevance Credibility Tech. cons Total grade Vehicle-km and GDP at constant price 9,33 9,00 10,00 28,33 Population exposure to air pollution by particulate matter 7,67 9,00 10,50 27,17 Sufficiency of Member States’ proposals for protected sites under the habitats directive 6,67 9,00 11,00 26,67 Land-use change, by category 7,00 6,50 12,00 25,50 Generation of waste by all economic activities and by households 6,00 9,00 10,50 25,50 Contribution of the clean development mechanism (CDM) to GHG emission reductions in developing countries 4,33 9,50 11,00 24,83 Index of apparent consumption of chemicals, by toxicity class 10,33 4,50 8,50 23,33 Exceedance of critical loads of acidifying substances and nitrogen in sensitive natural areas 7,67 6,00 7,50 21,17 Total scores of the selected “to be developed “ SDIs

25. Indicator Cluster criteria - grades Relevance Credibility Priority score Biodiversity index 11.67 4.00 7.67 Child wellbeing 7.33 0.00 7.33 Index of apparent consumption of chemicals, by toxicity class 10.33 4.50 5.83 Environmentally weighted indicator of material consumption 8.33 3.00 5.33 Green public procurement 8.67 4.00 4.67 Deaths due to infectious food-borne diseases 9.33 5.00 4.33 Socioeconomic disparities in health 6.67 3.50 3.17 Unmet needs for healthcare, by cause 5.67 3.00 2.67 Life years lost through premature death 6.33 4.00 2.33 Indicators proposed for further development within the INDI-LINK project

26. Next steps • Review of state-of-the-art of selected SDIs • Connecting with ongoing activities • Proposal how to contribute to the long-term methodological developemnt of the SDIs (October 07) • May 2008 – results (methodological development, pilot calculations)

27. Example – Total material consumption • Status quo: 2 main methodological approaches: • LCA-oriented (product level) • Input-Output analysis (sector level) • Planned work: • Evaluation of differences in results on national level (case studies) • Integration of the 2 approaches • LCA: basic commodities • I-O analysis: higher manufactured products and services • Expected outcome: • Pilot methodology • Pilot data for selected EU countries

28. Assessment of interlinkages between different priorities of the EU SDS (WP 2) • Review and comparative analysis of advantages and limits of different analytical frameworks for analysing interlinkages between different SD dimensions • Quantitative analyses of past interrelations applying selected analytical tools and indicators • Further development of tools and models to provide forecasts of selected SD indicators and an ex-ante assessment of future interlinkages

29. Thank you ! tomas.hak@czp.cuni.cz