Forecasting - 06-final

1. Forecasting: Principles and Practice Rob J Hyndman and George Athanasopoulos Monash University, Australia Chapter 6 - Judgmental forecasts Dr. Sasan Barak’s Book Reading Series Winter 2021 1

2. OUTLINE 6.1 Beware of limitations 6.2 Key principles 6.3 The Delphi method 6.4 Forecasting by analogy 6.5 Scenario forecasting 6.6 New product forecasting 6.7 Judgmental adjustments 6.8 Further reading 2

3. Judgmental Forecasting Is The Only Option 6 ★When there is a complete lack of historical data ★When a new product is being launched ★When a new competitor enters the market ★During completely new and unique market conditions Forecasting: Principles and Practice 3

4. Example 6 Australia was the first country in the world to introduce plain packaging for tobacco products. Tobacco companies were prohibited by law to use brand logos, colors or promotional text on their tobacco packaging Forecasting: Principles and Practice 4

5. Example 6 Judgment must be applied in order to forecast the effect of such a policy, as there are no historical precedents Forecasting: Principles and Practice 5

6. Is there any other situations? 6 Judgmental forecasting is used: When data are incomplete, or only become available after some delay Central banks include judgment when forecasting the current level of economic activity, a procedure known as nowcasting, as GDP is only available on a quarterly basis Forecasting: Principles and Practice 6

7. When does accuracy of judgmental forecasting improve? 6 The forecaster has: ★ Important and broad domain of knowledge ★ Up-to-date information Forecasting: Principles and Practice 7

8. When does accuracy of judgmental forecasting improve? 6 ★ The acceptance of judgmental forecasting as a science has increased, as has the recognition of its need ★Implementing well-structured and systematic approaches = improvements in judgmental forecasting Forecasting: Principles and Practice 8

9. Three General Settings 6 No available data, statistical methods are not applicable, and judgmental forecasting is the only feasible approach 1 Data are available, statistical forecasts are generated, and these are then adjusted using judgment Data are available and statistical and judgmental forecasts are generated independently and then combined 2 3 Forecasting: Principles and Practice 9

10. Three General Settings 6 ★When data are available, applying statistical methods is preferable and should always be used as a starting point ★Statistical forecasts are generally superior to generating forecasts using only judgment Forecasting: Principles and Practice 10

11. • 6.1 Beware of Limitations 11

12. Beware of Limitations 6.1 Judgmental forecasts are subjective, and therefore do not come free of bias or limitations ★Can be inconsistent Unlike statistical forecasts, which can be generated by the same mathematical formulas every time Forecasting: Principles and Practice 12

13. Beware of Limitations 6.1 ★Depends on human cognition and are vulnerable to its limitations For example, a limited memory may render recent events more important than they actually are and may ignore momentous events from the more distant past Forecasting: Principles and Practice 13

14. Beware of Limitations 6.1 ★Depends on human cognition and are vulnerable to its limitations A limited attention span may result in important information being missed A misunderstanding of causal relationships may lead to erroneous inferences. Forecasting: Principles and Practice 14

15. Beware of Limitations 6.1 ★Judgments can be varied due to the effect of psychological factors For example, one can imagine a manager who is in a positive frame of mind one day, generating forecasts that may tend to be somewhat optimistic, and in a negative frame of mind another day, generating somewhat less optimistic forecasts Forecasting: Principles and Practice 15

16. Beware of Limitations 6.1 ★Judgment can be clouded by personal or political agendas, where targets and forecasts are not segregated For example, if a sales manager knows that the forecasts she generates will be used to set sales expectations (targets), she may tend to set these low in order to show a good performance or exceed the expected targets Forecasting: Principles and Practice 16

17. Beware of Limitations 6.1 ★Even if forecast and target are well segregated, judgment may be affected by optimism or wishful thinking For example, it would be highly unlikely that a team working towards launching a new product would forecast its failure Forecasting: Principles and Practice 17

18. Beware of Limitations 6.1 ★Effect of anchoring (Anchoring Bias) The subsequent forecasts tend to converge or be close to an initial familiar reference point For example, it is common to take the last observed value as a reference point Forecasting: Principles and Practice 18

19. Beware of Limitations 6.1 ★Effect of anchoring Tversky and Kahneman (1974) define Anchoring to be when People make estimates by starting from an initial value that is adjusted to yield the final answer Adjustments are typically insufficient Different starting points yield different estimates, which are biased towards the initial values • • • Forecasting: Principles and Practice 19

20. Beware of Limitations 6.1 ★Effect of anchoring The forecaster is influenced by prior information, and therefore gives this more weight in the forecasting process Forecasting: Principles and Practice 20

21. Beware of Limitations 6.1 ★Effect of anchoring Anchoring may lead to conservatism and undervaluing new and more current information, and thereby create a systematic bias Forecasting: Principles and Practice 21

22. • 6.2 Key Principles 22

23. Key Principles 6.2 ★Set the forecasting task clearly and concisely ★Implement a systematic approach ★Document and justify ★Systematically evaluate forecasts ★Segregate forecasters and users Forecasting: Principles and Practice 23

24. Set the forecasting task clearly and concisely 6.2 ★ When setting the forecasting tasks, care is needed ★ Be clear about what the task is ★ Avoid ambiguous and vague expressions ★ Avoid emotive terms and irrelevant information ★ Conduct a preliminary round of information gathering before setting the forecasting task Forecasting: Principles and Practice 24

25. Implement a Systematic Approach 6.2 ★ Forecast accuracy and consistency can be improved by using a systematic approach ★ Involving checklists of information which are relevant ★ Identify what information is important and how this information is to be weighted ★ Devote significant effort and resources to put together decision rules that will lead to the best possible systematic approach Forecasting: Principles and Practice 25

26. Implement a Systematic Approach 6.2 When forecasting the demand for a new product, what factors should we account for and how should we account for them? Price Environment Competition ? ? Quality ? ? ? Quantity ? ? ? Forecasting: Principles and Practice 26

27. Document and Justify 6.2 ★Formalizing and documenting the decision rules in the systematic approach can promote consistency, as the same rules can be implemented repeatedly ★Requesting a forecaster to document and justify their forecasts leads to accountability, which can lead to reduced bias ★Formal documentation aids significantly in the systematic evaluation process that is suggested next Forecasting: Principles and Practice 27

28. Systematically Evaluate Forecasts 6.2 ★Monitoring the process and identify unforeseen irregularities ★Environment is dynamic, Changes occur, and you need to monitor these ★keep records of forecasts and use them to obtain feedback when the corresponding observations become available ★Feedback and evaluation lead to the more accuracy Forecasting: Principles and Practice 28

29. Segregate Forecasters and Users 6.2 ★ Forecast accuracy may be decreased if the forecasting task is carried out by users of the forecasts ★ Forecasting is about predicting the future as accurately as possible, given all of the information available, including historical data and knowledge of any future events that may impact the forecasts ★ Forecasters and users should be clearly segregated! Forecasting: Principles and Practice 29

30. Segregate Forecasters and Users 6.2 For a new product, the forecast should be a reasonable estimate of the sales volume of a new product, which may differ from what management expects the sales will be ★Explain and clarify the process and justify the basic assumptions to potential users ★The way in which forecasts may then be used and implemented depend on managerial decision making ★Set targets is different from producing forecasts, and the two should not be confused Forecasting: Principles and Practice 30

31. Segregate Forecasters and Users 6.2 For example, management may decide to adjust a forecast upwards as the forecast may be used to guide purchasing and stock keeping levels Such a decision may be taken after a cost-benefit analysis reveals that the cost of holding excess stock is much lower than that of lost sales This type of adjustment should be part of setting goals or planning supply, rather than part of the forecasting process. If forecasts are used as targets, they may be set low so they can be exceeded easily Forecasting: Principles and Practice 31

32. Example: Pharmaceutical Benefits Scheme (PBS) 6.2 Prescription medicines subsidies in Australia in 4 categories: ★ Concession Copayments ★ Concession Safety Net ★ General Copayments ★ General Safety Net Each person makes Concession Copayment per PBS medicine ($5.80) Concession Safetythreshold = ($348) For the rest of the financial year, all PBS- listed medicines are free. Each patient makes General Copayment per PBS medicine ($35.40) General Safety Netthreshold = ($1,363.30) For the rest of the financial year, they contribute a small amount per PBS-listed medicine ($5.80) Forecasting: Principles and Practice 32

33. Example: (PBS) – Forecasting Process 6.2 Judgmental forecasts are generated for : New listings of medicines and Estimating the impact of new policies The PBS forecasting: ● 84 groups of PBS-listed medicines ● Forecasts in each of the four PBS categories, a total of 672 series ● Setting the government budget allocated to the PBS, which is over $7 billion per year, or 1% of GDP Forecasting: Principles and Practice 15 33

34. Example: (PBS) – Forecasting Process 6.2 Judgmental forecasts for new listings of medicines and for estimating the impact of new policies The data used which were obtained from various government department Things that are calculated from the data provided Forecasting: Principles and Practice 34

35. Example: (PBS) – Forecasting New Listings 6.2 Asked companies to submit detailed forecasts The Pharmaceutical Benefits Advisory Committee provides systematic approach for generating these forecasts This structured process helps to reduce the likelihood and effects of deliberate self-serving biases Two detailed evaluation rounds of the company forecasts are implemented by a sub-committee, one before the medicine is added to the PBS and one after it is added Finally, comparisons of observations versus forecasts for some selected new listings are performed, 12 months and 24 months after the listings Forecasting: Principles and Practice 35

36. Example: (PBS) – Policy Impact Forecasts 6.2 ★ There were no systematic procedures for policy impact forecasts ★ On many occasions, calculated by a small team, and were often heavily reliant on the work of one person ★ The forecasts were not usually subject to a formal review process ★ There were no guidelines for how to construct judgmental forecasts ★ There was often a lack of adequate documentation about how these forecasts were obtained, the assumptions underlying them, etc Forecasting: Principles and Practice 36

37. Example: (PBS) – Hyndman Recommended Changes 6.2 -The guidelines for forecasting new policy impacts be developed, to encourage a more systematic and structured forecasting approach -The forecast methodology be documented in each case, including all assumptions made in forming the forecasts -The new policy forecasts be made by at least two people from different areas of the organisation -A review of forecasts be conducted one year after the implementation of each new policy by a review committee. The review committee should include those involved in generating the forecasts, but also others Forecasting: Principles and Practice 37

38. • 6.3 The Delphi Method 38

39. The Delphi Method 6.3 Was invented by Olaf Helmer and Norman Dalkey in the 1950s in purpose of addressing a specific military problem It assumes forecasts from a group are generally more accurate than those from individuals The aim of the Delphi method is to construct consensus forecasts from a group of experts in a structured iterative manner. A facilitator is appointed in order to manage the process Forecasting: Principles and Practice 39

40. The Delphi Method Workflow 6.3 1. A panel of experts is assembled 2. Forecasting tasks are set and distributed to the experts 3. Experts return initial forecasts and justifications. These are compiled and summarized in order to provide feedback 4. Feedback is provided to the experts. This step may be iterated until a satisfactory level of consensus is reached 5. Final forecasts are constructed by aggregating the experts’ forecast Forecasting: Principles and Practice 40

41. Experts and Anonymity 6.3 ★ Identify a group of experts who can contribute to the forecasting task ★ The usual suggestion is somewhere between 5 and 20 experts with diverse expertise ★ Experts submit forecasts and also provide detailed qualitative justifications for these Forecasting: Principles and Practice 41

42. Delphi Key Factors 6.3 A key feature of the Delphi method is that the participating experts remain anonymous at all times All experts are given an equal say and all are held accountable for their forecasts No need to meet as a group in a physical location! Forecasting: Principles and Practice 42

43. Setting the forecasting task in a Delphi 6.3 ★ Conduct a preliminary round of information gathering from the experts before setting the forecasting tasks ★ As experts submit their initial forecasts and justifications, valuable information which is not shared between all experts can be identified by the facilitator when compiling the feedback Forecasting: Principles and Practice 43

44. Feedback 6.3 Feedback Includes summary statistics of the forecasts and outlines of qualitative justifications As the feedback is controlled by the facilitator, there may be scope to direct attention and information from the experts to areas where it is most required Forecasting: Principles and Practice 44

45. Iteration 6.3 Repetition of the process of submitting, receiving feedback, and reviewing the forecasts until reaching a level of satisfactory ★ Usually two or three rounds are sufficient ★ The final forecasts are usually constructed by giving equal weight to all of the experts’ forecasts Forecasting: Principles and Practice 45

46. Limitations and Variations 6.3 ★ Delphi method can be time consuming ★ If it is taking a long time to reach a consensus, the panel may lose interest and cohesiveness ★ “estimate-talk-estimate” method is when the experts can interact between iterations, while forecast submissions still remain anonymous Forecasting: Principles and Practice 46

47. The Facilitator 6.3 ★ Responsible for the design and administration of the process ★ Responsible for providing feedback to the experts and generating the final forecasts ★ Responsible for arguing important information The facilitator needs to be experienced enough to recognize areas that may need more attention, and to direct the experts’ attention to these The efficiency of the facilitator can increase the probability of a successful Delphi method Forecasting: Principles and Practice 47

48. • 6.4 Forecasting By Analogy 48

49. Forecasting by Analogy 6.4 A useful judgmental approach which is often implemented in practice is forecasting by analogy A common example is the pricing of a house through an appraisal process Forecasting: Principles and Practice 49

50. Example: Designing a High School Curriculum 6.4 A small group of academics and teachers were assigned the task of developing a curriculum for teaching judgment and decision making under uncertainty for high schools in Israel How long it would take for the curriculum to be completed? Forecasting: Principles and Practice 50