Flaw of Averages

1. Flaw of Averages • This presentation explains a common problem in the design and evaluation of systems • This problem is the pattern of designing and evaluating systems based on the “average” or “most likely” future projections • Problem derives from misunderstanding of probability and systems behavior, known as FLAW OF AVERAGES

2. Flaw of Averages • Name derives from Sam Savage • It is a pun, integrating two concepts It refers to • A mistake => a “flaw” • The concept of the “law of averages”, that is, that things balance out “on average” • The flaw consists of assuming that design or evaluation based on “average” (or most likely) conditions give correct answers

3. A motivating example The design of an oil platform and wells in Golf of Mexico (Babajide)

4. Gulf of Mexico Platform Probability Mass Functions Note: “Most likely” scenarios are 150 and 100

5. Combined PMF

6. Comparison of Values Based on “most likely” estimates Based on actual distribution of possibilities Actual ENPV  Value based on Mostly Likely Conditions

7. Another motivating example Decision Analysis example

8. Comparison of Results Value based on most likely event (No Carbon Tax) = 6.00 Value based on recognizing possibility of Carbon Tax is different = 10.8

9. Why does Flaw occur? • Flaw is a pattern in systems design, Why? Several reasons converge • Difficult to evaluate system over many different possibilities – hard enough to create one design • Management fixes parameters (such as oil price) to facilitate comparisons in company • Uncertainties exist outside of technical specialty (markets, geology…) so that designers use “best estimates”

10. Mathematics of Flaw • Jensen’s law: • The Average of all the possible outcomes associated with uncertain parameters generally does not equal • the value obtained from using the average value of the parameters E [ f(x) ]  f [ E(x)] except when f(x) linear

11. Consequences • In simple terms, this means that the answer you get from a realistic description differs – often greatly – from the answer you from using most likely estimates • This is because the gains when things do well do not balance the losses when things do not • (sometimes they’re more, sometimes less) • In short: system behavior is non-linear

12. 3 Reasons for Non-Linearity • System response is non-linear • System response involves some discontinuity (step change) • Management rationally imposes a discontinuity

13. System Response is Non-Linear • Economies of Scale: Unit costs decrease with scale of production • Large initial costs prorated over volume, so that unit costs decrease as scale increases toward capacity • Increasing marginal costs as scale increases (labor, material costs higher) Unit Cost This is Usual Situation! Scale

14. System Response involves some Discontinuity Discontinuities = special form of non-linearity Discontinuities are Common: • Expansion of a Project might only occur in large increments (new runways, for example) • A System may be capacity constrained, so that profitability or values increases with demand up to a point, and then levels off

15. Management Creates Discontinuity • Whenever the Managers or System Operators decide to take some major decision about a project – to enlarge it or change its function – this creates a step change in the performance of the system. • This can happen often – and does! • See “Flaw of Averages” draft chapter

16. Take-Aways • Do not be a victim of Flaw of Averages • Do not value projects or make design decisions based on average or most likely forecasts. • Do consider, as best you can, the entire range of possible events and examine the entire distribution of consequences.