Transportation Project Evaluation and Prioritization: Exploring the Oregon Mosaic Analytic Tool

1. Transportation project evaluation and priority-setting: the Mosaic analytic tool Samuel Seskin Transportation Consultant Urban Sustainability Accelerator

2. Here is the story: • You should use two, complementary measures of value to make your decisions. • Adequate methods and data exist; the challenge is when to use them, so that... • information will inform but not dictate decisions.

3. Introduction to Oregon Mosaic • Developed by the Oregon Department of Transportation • Based on methods used over many years • Strong foundation in state of the practice performance measures • Fully web-accessible at www.oregonmosaic.org

4. Mosaic supports a central step in the planning process

5. Overall Goals of Using Mosaic • Inform decision makers about the mix and kinds of value associated with transportation investments and expenditures • Identify the most cost effective solutions given the community’s goals

6. Part 1: How to measure value subjectively

7. Do engineers use all the right measures?

8. Do planners have all the answers?

9. Is stakeholder engagement easy?

10. Is anything missing from political decisions?

11. We agree on the goals (probably)…

12. …but not their importance

13. The Weighting Process • Weighting is done by stakeholders • Stakeholders can reach agreement on how to “spend” 100 points among the categories

14. Try this: • Imagine you were buying a car. How much weight would you put, in advance, on price as a factor in your decision, versus color?

15. Now consider this: $17,000 $17,100 Does the fact that the difference in price is very small change how much weight you give that indicator in your final decision?

16. Weighted scores (Oregon Test Case)

17. Value-informed decision making: • Supports learning, not debating • Encourages discussion and exploration of value and values • Decisions are more transparent and defensible. • Results inform but do not dictate decisions.

18. Value-informed decision making: Summary • Confess your values • Do this several times—when you are starting the evaluation process, and then again after you get the results. • Stakeholders should debate values with fellow stakeholders. • Test them against the data, to see the effect of the importance you put on them, on the overall outcome or decision.. Often, they matter less than you think.

19. QUESTIONS AND DISCUSSION

20. Part 2: How to measure value in dollars

21. We agree on the goals (probably)…

22. Two ways of measuring value: Value in dollars Value informed by stakeholders

23. Why measure value in dollars? • Because they are scarce. • Because they enable us to compare value more easily than using doughnuts. (Do you value chocolate cream more than glazed? glazed more than frosted?)

25. Illustrative indicators measured in dollars • MO.1 - Travel Time • MO.3 - Reliability (Recurring congestion) • MO.4 - Reliability (Non-recurring congestion) • MO.5 - User Costs • EV.2 - Changes in transportation costs by industry (business travel and freight) • EV.4 - Changes in productivity from increased connectivity • ES.1 - Criteria Air Contaminants • ES.4 - Life-cycle CO2e • FT.1 - Capital Costs • FT.2 - Other Lifecycle Costs • FT.3 - Total Revenue • SA.1 - Fatal, Injury A, and Injury B Crashes • QL.1 - Lives saved due to active transportation • QL.2 - Reduced incidence of diseases due to active transportation • QL.3 - Quality of the travel environment • QL.4 - Noise Impacts

26. Indicators of livability and quality of life

27. Indicators of economic vitality

28. Sources of Benefit/Cost Ratio (Oregon Test Case)

29. Measuring value in dollars: Summary • It can be done! • Acknowledge a range of monetary values. • Test the effects of using different values on the results. • USA is the only English-speaking wealthy country, and one of the few wealthy countries, where BCA is not a routine part of transportation decision making.

30. Measuring value in dollars: Summary • Use the best available information within the constraints of time and money. • When you are in the swamp, apply insect repellent.

31. QUESTIONS AND DISCUSSION

32. Part 3: How to compare the two measures of value

33. This was a test…

34. Comparison of stated value to $benefit-cost ratio(Oregon Test Case)

35. Value trade-offs

36. Best Practice Example: Metropolitan Transportation Commission

37. Best Practice Example: Metropolitan Transportation Commission

38. Best Practice Example: Metropolitan Transportation Commission

39. Best Practice Example: Metropolitan Transportation Commission

40. Best Practice Example: Metropolitan Transportation Commission

41. Five myths about value-based analysis: • These methods will constrain political imperatives. • These methods will work against (or work for) my constituency’s interests. • These methods add time and needless complexity to decision making. • The methods aren’t ready for “prime time”. • Some things just can't be measured, or certainly not measured well; these methods reduce everything to numbers.

42. The case for using two measures of value • It can be done! • It offers a way forward for stakeholders of all persuasions. • It does not dictate decisions. • It supports learning and reduces the need for debating.

43. What’s useful about this approach? • It offers a more comprehensive set of information to decision makers. • It can be applied to scenarios, projects, systems or asset classes. • It doesn’t change who makes decisions; it better informs the decisions.

44. What have we learned from applications to date? • The methods and data exist; the challenge is how and when to use them, because technical staff (internal stakeholders) are often the biggest source of resistance. • Like value engineering, the information enables exploration of tradeoffs between projects as well as project features. • Remind yourselves and your elected officials that no matter what they fear, you know that politics always trump analytics.

45. QUESTIONS AND DISCUSSION

46. THANK YOUSam Seskinsseskin@Comcast.net

48. IT’S NOT ABOUT THE MEASURES.IT’S ABOUT HOW YOU USE THEM. ADDITIONAL SLIDES

49. Ways to Improve Decision making • Open the black box. Provide adequate documentation • Peer-review the tool. • Make the tool stakeholder-friendly. Strong graphics Real-time learning opportunities • Make the methods scalable. • Acknowledge uncertainty.

50. Ways to Improve Decision making 6. Measure value in multiple ways. 7. Unbundle the components of value. 8. Specify measurement methods clearly to minimize “advocacy”. 9. “Open in Safe Mode”. 10. Use it to shoot the injured animals first.