Environmental Systems Analysis (ESA 20506/22806)

1. Environmental Systems Analysis (ESA 20506/22806) • Multi Criteria Analysis • Carolien Kroeze and Karen Fortuin

2. Content of this Presentation • What is a Multi-Criteria Analysis (MCA)? • Purpose and characteristics of a MCA • How to do a MCA? • How to apply a MCA in your group?

3. What is Multi Criteria Analysis? • Definition (CIFOR, 1999): • “Multi Criteria Analysis is a decision-making tool, developed for complex multi-criteria problems that include quantitative and/or qualitative aspects of the problem in the decision making process.”

4. Purpose of MCA • In environmental systems analysis we want to provide decision makers with information on the consequences of alternative solutions to complex problems • MCA is used to compare and rank alternative options or courses of action, taking into account and evaluating their respective consequences

5. Multi Criteria Analysis (MCA) • But: What are possible criteria based on which these alternative solutions should be evaluated? • Examples of such criteria are • the costs of the possible solutions • the effect on the environment • the socio-economic consequences • negative side-effects on other issues

6. An Example • Two thirsty people are trying to decide whether to buy a can of Cola or a bottle of Orange Juice. • Criteria: costs and health benefits • Person 1 is concerned by the small amount of money they have and wants to buy the cheaper Cola • Person 2 is concerned with living a long healthy life and is willing to pay for the healthier Orange Juice

7. Environmental Systems Analysis: example • European governments want to compare alternative air pollution policies • or acidification in Europe • What are criteria to evaluate alternative policies? • the overall European costs of the emission control • the costs for specific countries or sectors • the short term and long term effect on ecosystems • the socio-economic consequences of these policies • negative side-effects on eutrophication, smog, or the greenhouse effect • There are many possible criteria, but they may all lead to different • preferred solutions!

8. Environmental Systems Analysis: example • The Dutch government asked an environmental research institute to analyze the environmental consequences of integrated farming compared to organic farming • What are possible criteriato evaluate the environmental impact of the production of potatoes? • Emissions of ammonia (NH3) to the air causing acidification • Losses of nitrate (NO3-) causing groundwater pollution • Losses of biocides causing toxicity problems • There are at least three criteria, but they may lead to different solutions

9. Emissions in kg, per kg potato NB: hypothetical case! Three criteria Nitrate Ammonia Biocides (kg /y) (kg /y) (kg /y) Integrated 4.1 0 0.4 farm Organic 5.3 1.0 0 farm Green : Most environmental friendly

10. Pair-wise comparison of the two types of farming • Criterion Most environmentally friendly • Nitrate Integrated farming • Ammonia Integrated farming • Biocides Organic farming • Conclusion: without weighing the criteria we cannot say which type of farming is most environmentally friendly!

11. Characteristics of MCA • Criteria have different dimensions • e.g. costs, deposition levels, area of damaged ecosystems • Criteria differ in weight • e.g. the critical loads for acidification may be exceeded to a larger extent than the targets for eutrophication • Weights depend on ‘vision’ • e.g. people may consider one environmental problem more urgent than another • Qualitative and quantitative information • e.g. quantitative emissions loads approach versus qualitative opinions on the environment • About 10 different MCA methods available

12. Procedure for MCA • Establish the decision context • Identify the options • Identify criteria • ‘Scoring’ • ‘Weighting’ • Combine the weights and scores for each option to derive an overall value • Examine the results • Sensitivity analysis • (from: ODPM, 2005)

13. Iteration to consider constraints, objectives or criteria Forecasting Future context Iteration to improve the predictive process Initialisation Formulating the problem Boundaries & constraints Alternatives Identifying, designing and screening alternatives Building & using models for predicting consequences Comparing & ranking alternatives Objectives Consequences (impacts) Communicating results Values & criteria Iteration to improve alternatives Iteration to reformulate the problem MCA compared with Six-step method(Findeisen & Quade, 1997) Step 4, 5, 6 Step 1 and 2 Step 7, 8 Step 3

14. Identify criteria and sub-criteria (step 3) • (See ODPM, 2005) • What would distinguish between a good solution and a bad one? • Once you have a set of criteria try to group them (e.g. costs, effect on the environment, socio-economic consequences, side-effects on other issues) • Example: criteria to appraise the different solutions to a transport problem (from DETR, 1998)

15. Example Identify Criteria • Five main objectives (i.e. criteria) for transport: • To protect and enhance the natural and built environment; • To improve safety for all travellers; • To contribute to an efficient economy, and to support sustainable economic growth in appropriate locations; • To promote accessibility to everyday facilities for all, especially those without a car; • To promote the integrationof all forms of transport and land use planning, leading to a better, more efficient transport system.

16. Example Identify Sub-criteria (indicators) • Noise • Local air quality • Landscape • Biodiversity • Heritage • Water • Public Transport • Severance • Pedestrian Environment Accessibility

17. ‘Value tree’

18. Assess the criteria • Completeness • Redundancy • Operationally • Mutual independence of preference • Double counting • Size • Impacts over time

19. ‘Scoring’ • Assess the expected performance (consequences) of each solution against each criterion. This can be done: • Quantitativelye.g. in monetary terms, number of accidents, increase of CO2 emission, etc. • Qualitativelye.g. in words‘no significant impact’ Result: Performance matrix

20. Example Performance Matrix

21. Example of part of a Performance Matrix

22. Scoring in a full MCA • Take care that the scores are comparable • Use the same sense of direction: usually better performance means a higher score (See e.g. problem with costs) • Use e.g. interval scales (e.g. 0 – 100) (see reader) • In your group work, you may use a scale 0-100 or 1-5.

23. Example Scoring Method • Assign 100 to the most preferred option • Assign 0 to least preferred option • Score the remaining options in comparison to these two (= relative judgment) 0 100 Least preferred Most preferred Relative strength of preference

24. Example Scoring Table

25. Example Scoring Method (CIFOR)

26. ‘Weighting’ • Assign a weight (value) for each of the criteria to reflect their relative importance. Different methods exist. • In reader: ‘Swing weighting’: The weight on a criterion reflects both • Range of difference of the option • How much that difference matters or how important that criterion is

27. Example assigning weights

28. Other Weighting Methods • Distance to target reflects the difference (or distance) between the calculated actual emissions of a certain pollutant and the desired (or target) level; “political basis” • ‘No significant adverse effect level’ (NSAEL) calculates the amount by which current emissions exceed the NSAELs; “scientific basis” • Panel method reflects the subjective opinion of a group of experts or stakeholders (e.g. CIFOR Method)

29. Examples of Weights or Valuation Factors

30. Calculate the overall preference scores • The overall preference score (Si) of option i is the sum of all weighted average scores on each criterion. • Si = w1si1 +w2si2 + …….. + wnsin = Σ wjsij sij = score for option i on criterion j wj = weight for criterion j n = amount of criteria taken into account n J = 1

31. Example calculating the overall scores Tefal Thickn Thin 8780 is the best

32. Group work and Weighting Methods • You may consider your group a panel that assigns the weights and use the following methods: • Ranking: assigning each criterion a rank that reflects its perceived degree of importance relative to the decision being made. The criteria can then be ranked (first, second, etc). • Rating: similar to ranking, but criteria are assigned “percentage scores” between 0 and 100, while the total scores for all criteria must add up to 100. • Source: CIFOR • http://www.cifor.cgiar.org/acm/methods/toolbox9.html

33. Two Types of Ranking • Ordinal Ranking: each expert is asked to put the list of decision elements in order of importance • Regular ranking: assigns each element relevant to the decision process a rank depending on its perceived importance.

34. Example Ordinal Ranking • Rank the following 3 environmental problems: • Global warming • Acidification • Eutrophication • Which is the most important, which is the least important? • Assign a “3” to the most important, a “1” to the least important and a “2” to the middle one.

35. Result: • Students in the past revealed the following order of importance for 3 environmental problems: • Global Warming • Acidification • Eutrophication

36. Regular Ranking • Regular ranking: assigns each element relevant to the decision process a rank depending on its perceived importance.

37. Regular Ranking • Example: Ranking of criteria to evaluate the sustainability of policies that affect forest management • Suppose a group of experts is asked to assign ranks to criteria using a 9 point scale

38. CIFOR Example • Four Criteria to evaluate the sustainability of forest management policies: • Funding: adequate, long-term funding for the forest management • Laws: legal framework protects forest resources and access • Buffer zones: a functional buffer zone exists • Forest use: reinvestments into forest-use options

39. Regular Ranking Possible ranking of criteria by one of the experts Advantage compared with ordinal ranking: possible to specify “grades” of importance, but may not be discriminating enough

40. Rating • Rating is a technique where each expert is asked to give each decision element a rating of a percentage score between 0 and 100. The scores for all elements must add up to 100. • Advantage: it provides both ordinal and cardinal measure of importance • Ordinal refers to order of importance • Cardinal refers to difference in magnitude between two criteria

41. Rating • Example: for the same criteria, one expert might give the following ratings

42. Ranking by a team of experts • “Laws” are considered the most important criterion by this team of experts

43. Rating by a team of experts • “Laws” are considered the most important criterion by this team of experts

44. Ranking and Rating by a team of experts • Suppose three experts rank and rate four criteria as follows:

45. Ranking and Rating by a team of experts • Sum of Votes = sum of votes by individual experts • Relative weight = (Sum of Votes / Total)*100 • Combined weight = (Relative weights rating + ranking) / 2

46. Calculate the overall preference scores • The overall preference score (Si) of option i is the sum of all weighted average scores on each criterion. • Si = w1si1 +w2si2 + …….. + wnsin = Σ wjsij sij = score for option i on criterion j wj = weight for criterion j n = amount of criteria taken into account n J = 1

47. Scoring the alternatives • Example: Scoring method developed for CIFOR

48. Scoring the alternative (i.e. forest management policy) • The experts assigned the following scores to the 4 selected criteria:

49. Overall evaluation of one alternative Final score = 281/100 = 2.81 = below acceptable

50. How to use MCA in your group work? • Evaluation of your alternative solutions (scenarios) • First, decide on the criteria that you want to use in your evaluation • Second, identify appropriate indicators for each of the criteria (more than one indicator for each criteria is OK) • Third, score the alternatives on each criterion based on their performance on that criterion