Behavioral Issues in Forecasting

1. Behavioral Issues in Forecasting Paul Goodwin University of Bath, UK

2. Why forecasting is important • Short term: production planning, staff work scheduling inventory control, materials purchasing, stock market decisions. • Long term – new production facilities, new infrastructure projects, training new staff. • Very long term –probably not worth forecasting: use methods like scenario planning.

3. All forecasts involve judgment… • Choice of forecasting method • How much past data to use • Which variables to include in your model etc. However, we are concerned here with the direct use of judgment in forecasting i.e. forecasts based wholly on judgment or judgmental adjustments to statistical forecasts

4. Role of direct judgment in forecasting • Judgmental extrapolation of time series • Judgment based on non-time series information

5. Role of direct judgment in forecasting • Judgmental adjustment of statistical forecasts • Probability judgments.

6. Topics • Ten cognitive biases in judgmental forecasting and methods that might overcome them. • Future research challenges.

7. Management judgment can bring benefits to forecasts 1. It can take into account special events: e.g. - a disease outbreak, -a government initiative - a sales promotion 2. It can compensate for a lack of relevant historical data.

8. Management judgment can bring benefits to forecasts 3. It can allow for situations where managers have some control over the variable to be forecast 4. It may produce more acceptable forecasts because: -managers have a sense of ownership of the forecasts - complex statistical methods may lack transparency and hence lack credibility…..

9. British Gas. Daily forecast of gas demand(Taylor & Thomas, 1982)

10. But judgment can also be problematical……

11. Test your judgment quiz Questions 1 to 3

12. Biases in judgmental forecasts and possible solutions

13. 1. Anchoring and adjustment • When making a forecast people often start with an initial value (the anchor) and then adjust from this. Danger Estimates tend to be tooclose to the anchor -even if it's an implausible value...

14. How old was Gandhi when he died? Source: Strack & Mussweiler One group asked was he older or younger than 9? Then asked to estimate his age at death Mean answer 50 Another group asked was he older or younger than 140? Then asked to estimate his age at death Mean answer 67 Correct answer: 78

15. Test your judgment quiz • Answer to question 1 Population of Egypt in 2006 = 72.6 million

16. Effect on forecasts of anchoring Anchoring can cause people to underestimate upward trends because they stay too close to the most recent value…..

17. Test your judgment quiz 2001 Q3: Actual =£4.323 billion Q2 Answer: Linear regression line would forecast £4.800 billion

18. Underestimation can be particularly severe for exponential trends…

19. A piece of paper is 0.193 mm (0.0076 inches) thick. If the paper could be folded 40 times, use your judgment to forecast the thickness of the folded sheet.

20. Answer Mathematics shows that after 40 folds the paper’s thickness will be: 0.193mm x 240 = 212,206 kilometres or 131,862 miles. That’s more than half the distance to the Moon It’s highly likely that you underestimated this!

21. Test your judgment quiz • Answer to question 3 • Sales = 0.6 et where t = year So in year 12 sales = 97,653 units

22. Solutions • Use statistical methods to extrapolate trends rather than judgment. 2. Increasing the accessibility of anchor-inconsistent knowledge mitigates the effect. -Considering the opposite (i.e., generating reasons why an anchor is inappropriate). Mussweiler et al. (2000)

23. Test your judgment quiz Question 4

24. 2. Overconfidence Overconfidence that an outcome will fall within an estimated range…. E.g. 90% judgmental prediction interval Lower limit Upper limit Most Likely Actual outcome • Not clear what causes overconfidence. Anchoring used to be the accepted explanation. • Some evidence of cultural differences in its extent (Meng et al. 2016).

25. Ben-David et al. (2013) asked a large sample of U.S. financial executives to produce 80% prediction intervals of one-year-ahead stock market values • They were so narrow that they only included the true values on 36.3% of occasions

26. Answers to “Test your judgment” Question 4 a) 6300 miles or 10,139 kilometres b) 31, 241, 030 c) 1938 by Biro d) 1876 e) 14000 feet or 4000 metres

27. Solution 1 • Augmentation: once you’ve estimated a range –double it! E. g. Initial 90% prediction interval for sales: 40 to 60 thousand units After Augmentation: 30 to 70 thousand units

28. Solution 2 • We are better at estimating a coverage probability for a given interval than we are at producing an interval to have a given probability E.g. What is the probability that the interval 45 to 55 captures Obama’s age when he became President? rather than.. Estimate a 90% interval for Obama’s age when he became President.

29. Test your judgment quiz Question 5

30. 3. Availability bias • Forecast is over influenced by recent, easily recalled or vivid events or events reported recently in the media…..

31. Study by Johnson et al. (1993) found people willing to pay more for airline travel insurance covering terrorist attacks than for deaths from all possible causes • Sales of earthquake insurance • are highest after an earthquake • when the risk is lowest. • They then decline as time passes • (while the risk gradually increases).

32. Test your judgment quiz Answer to question 5 In the USA in 2007 the proportion of people (aged 12 or over) who were victims of robbery was approximately 2 in every 1000 or 0.2% (Source US Dept of Justice)

33. Test your judgment quiz Question 6

34. 4. Seeing patterns in randomness

35. 6 monthly cycle in demand?

36. Rats handled randomness better than Yale students 60% of time 40% of time • Rat went to side where food appeared most frequently – correct 60% of time • Students tried to spot patterns – correct only 52% of time (see Tetlock, 2005)

37. 5. The narrative fallacy -People are brilliant at inventing explanations for these random movements “Our customers have been stocking up in anticipation of a price increase” “OK last time they didn’t stock up when they expected a price increase –but then interest rates were high so holding inventory was costly”

38. Even Nobel Laureates are not immune… Output per worker Robert Solow developed a hypothesis to explain this ‘structural shift’ in the US economy for 1943-1949 It turned out the anomaly was a result of mistakes in arithmetic Capital per worker

39. “Our new Harvard -educated Sales Manager is working miracles” • The dull statistical forecast can offer no competition to the colorful, but often groundless tales of why the graph has swung in a particular direction -and so it gets adjusted. The exponential smoothing forecast based on α = 0.15 predicts little change

40. Test your judgment quiz It’s best not to adjust the statistical forecasts as the deviations from it are purely random Answer to question 6

41. Solution • When identifying patterns in data, don’t believe that you can do a better job than a statistical forecasting system. • Easier said than done..

42. Key research challenge • How do we discourage managers from over interfering with statistical forecasts?

43. Many companies judgmentally adjust a large % of their statistical forecasts (Fildes et al., 2009) Evidence from four companies: Over 26000 SKUs

44. How large are the adjustments people make? Three Manufacturers Retailer 12% 8% 4% -100% 0% 100% 200% 0% 100% 200% Many small adjustments A few very large + adjustments

45. Size of adjustments Results from one company Small adjustments to software’s forecasts waste time and often reduce accuracy

46. Test your judgment quiz Question 7

47. 6. Optimism bias • So prevalent that UK’s Department for Transport has an optimism bias adjustment system ..though some of this attributable to ‘strategic misrepresentation’ or lying E.g. In an analysis of 258 transport infrastructure projects completed between 1927-1998 costs were underestimated in 9 out of 10 cases

48. One reason for optimism bias:Ignoring the base rate • We focus on the specific characteristics of a future event and ignore, or discount, the underlying statistical rate at which the event occurs…

49. “We are 90% certain this product will be profitable. • It’s been designed by an enthusiastic team. • It’s the biggest investment we’ve made in R & D • Competing products are nowhere near to being launched. • It’ll be backed by a huge advertising campaign” The base rate: only 35 % of products launched in this market make a profit.

50. Over optimism in the success of marriage? Base rates: • UK: 33% marriages between 1995 and 2010 ended in divorce (Source: Office of National Statistics) • USA: Divorce rate in America for third marriages is between 70% to 73% (Source: Forest Institute of Professional Psychology, Springfield)