Modelling in an Imperfect World

1. Planning and policy advice with less-than-rational human beings Modelling in an Imperfect World Luis Willumsen

2. Some old concerns • Our track record is not brilliant • Models are a simplification of reality based on some useful theoretical assumptions and sufficient data to estimate them • Models are valid insofar the theoretical assumptions remain reasonable;sadly, our theoretical assumptions do not represent human behaviour well • Therefore, model results are worth little without interpretation and judgement • So, what do we know about human beings and choices that can help us provide better advice? Concerns

3. Contents • The key underpinnings of transport demand modelling • How travellers really make decisions (the Kahneman model) • Two characters: System 1 and System 2 • Two selves: Experiencing and Remembering selves • Two species: Homo Economicus and Homo Sapiens • Three contexts for forecasting: • Policy Advice • Planning • Forecasting demand and revenue Contents

4. The four pillars of good models Key requirements • Behavioural choice modelling • Consistency of future behaviour Utility functions and choice models applied at different levels of aggregation The parameters in the utility functions and choice structures remain the same Tastes and preferences are given and stable, exogenous to our models Forecasting Appropriate feed-back through all relevant submodels to ensure consistent results Accurate allocation of populations and activities in the future Modelling growth and change based on cross-section data IS OK • System equilibrium • Good future population synthesis

5. The four pillars of good models Key requirements From this basis we build a picture of the future that enable us to compare alternative strategies, projects and policies on a like with like basis

6. Utility functions • Modelling choices • The ideal traveller • Is Rational, Selfish and its Tastes do not change, ever. • A "rational" being that considers opportunities and seeks to optimise his/her utility by careful choices. Behavioural models

7. The real life traveller • A partly rational but also emotional and collaborative being that: • Cares about changes more than absolute values • Cannot cope with too many options and uses heuristics • Has diminishing sensitivity to changes in utility • Is averse to losses • Does not react immediately Real travellers • But what do we know about this real traveller? • And how do we adapt our modelling and recommendations to him/her?

8. The experiencing self • This is the traveller while travelling • Experience a combination of good and bad aspects of travel • The remembering self • This is what the traveller remembers • Usually salient aspects of the journey • The end of the trips and the results are paramount Real travellers Subsequent decisions are more influenced by what is remembered than the actual journey itself

9. System 1 thinking • Intuitive, fast, automatic • Uses heuristics, often answering an easy question rather than a difficult one • Sensitive to changes • Assumes that what you see is all there is WYSIATI • Thoughtful, Logical, requires effort • Lazy, first tendency is to endorse System 1 • Can interact with S1 and train it • iPad and cover cost £550 • iPad costs £500 more than cover • How much is the Cover? • System 2 thinking Two aspects of human decision making

10. The Rational Human Being, Homo Economicus • There is strong suspicion that it is a convenient assumption but does not correspond to reality • This mismatch may matter less to develop theory but it does affect the forecasts and advice we provide • We are not truly Utility Maximisers.. • And we cannot consider all our alternatives.. • We are more affected by changes than by absolute values • And these changes are based on what we remember from previous experiences, for example delay or price Homo economicus

11. Kahneman-Tversky’s Prospect Theory Prospect Theory • Evaluation is relative to a neutral reference point (status quo) • Diminishing sensitivities to change (Compare £100 to £200 and £1900 to £2000) • Loss aversion; loses are more onerous than the respective gains

12. there are also other problems with our current models

13. Fussy prices and money • Santiago tags and gantries • A problem with money... • Separating use from payment crates a different type of money • Ignoring the different kinds of money and prices in our models will lead to wrong forecasts (probably underestimations) and poor advice. Electronic payment

14. Change job or residence • A problem with time.. • Our equilibrium models assume all changes happen at the same time • But people cannot instantly change jobs or homes, and not even time of travel. • We need to recognise the lags in behaviour • But we know and understand little about them • Mode or time of travel Lags in behavioural responses

15. Some possible improvements • Hierarchical structure of choices is important for some known biases: • One could give much more important weights to certain attributes in the case of elimination by aspects: Time first, etc.. • Nested choices Possible solutions In the case of asymmetric elasticities one can develop a special utility function, even non-linear; but this may create problems for convergence

16. Lags in behavioural change • Hierarchical structure of responses is purpose dependent(?) • For JTW HBShop • Route Route • Time of travel Destination • Mode Time of travel • Destination Mode • Frequency Frequency • Model a time horizon with some of these responses frozen and interpolate • Separate responses • But we know too little about these lags • Cross section data collection is poor at capturing these; this includes SP • We need to learn more from time series and from experimentation Possible solutions

17. Does all of this matter? • Not really, we only want to compare Plans/Schemes/Policies on like-with-like basis that we all agree is good enough • OK, it is not perfect, but after a little while people do change because of an accumulation of minor disruptions • There is a trade-off between behavioural accuracy and the equilibrium we need to compare schemes; we vote for consistent comparisons • BUT • Schemes or plans may affect different responses in different ways • Sometimes it is important to get the sequence of interventions right • When forecasting for concessions the right timing and the right response are paramount Does it matter?

18. Improving interpretation and judgment

19. So.... • Human nature limits the accuracy of our models • There are implications for Research and for an evolving Best Practice • For Research: • Develop a better understanding of how errors and accuracy is affected by the level of disaggregation of our models and data • Identify lags in behavioural change and develop best ways to deal with them (more social psychology and less mathematics and computational efficiency perhaps?) • Develop a better relationship between objective (generalised cost) change and perceived loss/gain • Understand how people switch between System 1 and System 2 modes of thinking in the context of travel Implications

20. NEW DATA SOURCES WILL HELP • Use of mobile phone, bluetooth, smart card and GPS data • To monitor performance • To infer trip matrices • To study experiments Smartcards + GPS

21. Taxi Traffic in Vienna Mobile phone

22. Mobile phone data Mobile phone data

23. Location based on mobile phone cells Mobile phone data

24. Basic principles • Annonymised data • Great potential to study behaviour with large samples and OVER TIME • Take advantage of natural “experiments” Mobile phone data

25. General recommendations on forecasting practice • Our business is not modelling but forecasting • We need transport models but our existing tools are less reliable than we pretend; we must acknowledge uncertainty and risk from the outset • We should start experimenting with the careful adaptation and use of existing techniques to account for more realistic behaviour • Interpretation and judgement, professional responsibility, should be more open and transparent • Design and undertake experiments whenever possible, to improve and mediate model results; and this is easier now than in the past • Document experience more openly • How much influence can be applied to the future? Forecasting

26. Influencing the future • Governments • Policies regarding parking, fares, competition • Competing schemes • Taxes and subsidies • Private sector operator (Toll Road, Public Transport…) • Marketing tools • Selective pricing (peak/off-peak, promotions, discounts..) • Sometimes… • Joint development • Contractual or negotiated commitments to limit competition • Changing and upgrading the “offer” (extensions, premium services..) Influence and malleability

27. Influencing the future Influence and malleability

28. Policy advice • Not always depending on modelling • But our experience should be valuable as it would add analytical rigour to policy discussions • For example, the issue of Fuel Taxation vs. Road User Charges • Identification of winners and losers will be more central • Should experiment with the production of psychological impact evaluation in addition to “objective accounts” • The role of other “difficulties” of payment, information, familiarity, WYSIATY • Engage in the discussion of implementation, communication, sequencing and timing (remembering and experiencing self, S1 and S2 thinking modes) Policy advice

29. Transport Planning emerging practice • Use conventional tools but allow for lags in responses; even with assumed lag rates • This requires models where certain responses can be switched off at will • Show and discuss the impact of these lags and if critical look for other approaches to settle the choice of plan/scheme • Identify winners and losers and by how much • Account separately for large and small loses/gains • Acknowledge uncertainty and the risk of over-calibration and spurious precision Planning

30. Forecasting Traffic and Revenue • Our track record is better than that of bankers and regulators • But it is still not that good • Acknowledge uncertainty and risk from the outset: identify sources of risk, estimate their importance and focus on reducing them • Disaggregate for willingness to pay but do not over-complicate the model • Careful use of existing techniques, even with the limitations shown, is a reasonable approach. But, support forecasts from different complementary perspectives • For example, a classic model forecast, a trend extrapolation forecast and benchmarking against similar systems • Undertake risk analysis Forecasting

31. Round up • Some of these risk analysis techniques will also filter through into normal transport planning models and practice • Especially for key projects like High Speed Rail • Fundamental research into real travel behaviour and choices is necessary • Improvements to current practice that recognises some limitations of our models are possible and desirable • Benchmarking and well documented experience elsewhere will be used more often to support forecasts • This will be facilitated by new data sources and electronic trails • Modellers should engage more with real issues and develop reliable judgement and interpretation skills; this may require adaptation of training programmes Round up

32. Thank you