Portfolio Management Using Questionable Quality Data SPE 90395 Jim DuBois Portfolio Decisions Inc. September 28, 2004

1. Portfolio Management Using Questionable Quality DataSPE 90395Jim DuBoisPortfolio Decisions Inc.September 28, 2004

2. Opening Thoughts • This paper is not about improving data, but about how to work with the data you have- today. • Until you work with your data in a holistic manner, it is very unlikely that it will improve. • Treat data quality as another unknown, like reservoir size or Ps. Learn what can and what can’t hurt you, and to what extent. • May result in choosing a portfolio that is slightly less “optimal”, but also less sensitive to perceived data issues. SPE 90395

3. Avoid “Answerism” Many delay using advanced decision methods because “without perfect data, I won’t get the right answer”. But: • There is no “right” • There is no “answer” • There are, however, better insights, which lead to better decisions. SPE 90395

4. Better Decisions • The question facing you is not, “Can I make perfect decisions”, but can I make better decisions than I am making now? • You are going to make decisions. • Improve in cycles as you learn. SPE 90395

5. Procedure • Define the suspected data problem. • Describe the decision process. • Plan and execute the analysis. • Draw conclusions and communicate. SPE 90395

6. Step One- Define What You Don’t like About Your Data • Systemic Over or Under Estimation • Specific Bias • Random Error • Competence/Training/Consistency • Resources • Emphasis/Management Pressure SPE 90395

7. Step Two- Describe How the Data is Currently Used for Decisions • Used in PM, we just don’t trust it. • Used in another decision process (Hurdles, Rank and Cut, etc.) • Different data for approva, planning, budgeting, etc. • Used as a starting point, then “massaged” by decision makers, intermediaries. • Massaged how? • Used as a smoke screen to legitimize intuitive decision making. • Ignored SPE 90395

8. Step 2 Assumption For now, for today’s decision, this is the best the data is going to get. (Always try to improve for the next decision, but don’t abdicate today’s decision.) SPE 90395

9. Step Three- Plan Your Analysis Based on Answers to Two Previous Questions • Two types of consequence from questionable data: • Performance- If you act on a set of data which is flawed, the range of possible portfolio outcomes will be different (usually worse) than you expect. • Lost Opportunity- Had you had “correct” data, you would have made different decisions. This is often seen as a fairness issue. • Both types of consequence will always exist. SPE 90395

10. Step Three- Plan Your Analysis Based on Answers • Basic Procedure: • Analyze honoring current data. • Adjust data by increments in the direction feared. • Ps • Reserve Size • Capital Cost • Timing, etc. • (May be possible to use previous corporate behavior as a guide). Do not reoptimize yet. (Examples to follow). (con’t) SPE 90395

11. Step Three- Plan Your Analysis Based on Answers • Basic Procedure (continued): • Describe portfolio implications on two fronts: • Metric performance • Value • Reoptimize at an interesting sensitivity point • Describe implications on three fronts: • Metric performance • Selections • Value (con’t) SPE 90395

12. Step Three- Plan Your Analysis Based on Answers Basic Procedure (continued): • Examine implication of new selections when paired with original dataset. • Continue analysis, seeking a portfolio with acceptable performance if the data is “right”, but which has resilience if the data is “wrong”. • Can combine factors, but don’t overwhelm your ability to understand what is going on.Examine the impact of each factor separately first, then in combination. SPE 90395

13. Step Four- Draw Conclusions and Communicate Them • Did the feared data problem have the magnitude of impact you assumed? • In what time frame was the impact? • At what point did the impact become significant? • If there are other decision methods being used, use them with the same data variations. Are the impacts more or less than using the preceding analysis? SPE 90395

14. Step Four- Draw Conclusions and Communicate Them • If appropriate, reexamine at a coarser granularity. • Impactful changes vs. incidental • Changes that result in major shifts of capital, jobs, etc., obviously require more investigation • What was constant through the cases? • At what point (amount of bias) did impactful changes start? • Communicate impacts in both directions. SPE 90395

15. Examples Three Examples in the Paper: • Ps Sensitivity Analysis • Capital Sensitivity Analysis • Prod and Reserve Bias Analysis SPE 90395

16. Examples WE want to understand: • The effect of possible under-estimation on performance results • The effect of possible under-estimation on project selection • The level of under-estimation at which these effects become critical • If there exists a portfolio that performs better if there is under-estimation, but still performs well if there is not. SPE 90395

17. Base Data MetricsNPV = $18427 M SPE 90395

18. Base Data Selections SPE 90395

19. Ps –5% MetricsNPV = $17516 M SPE 90395

20. Ps –10% MetricsNPV = $16604 M SPE 90395

21. Ps –20% MetricsNPV = $14782 M SPE 90395

22. Ps –30% MetricsNPV = $12959 M SPE 90395

23. Ps –20% Metrics Reopt SPE 90395

24. Ps –20% Metrics (Compare)NPV= $14782 M SPE 90395

25. Ps-20% Selections SPE 90395

26. Ps –20% Selection Comparison SPE 90395

27. Ps –20% Selections- Original Ps SPE 90395

28. Ps Decrease Vs. NPV SPE 90395

29. Ps Sensitivity Conclusions • Not as sensitive to Ps as suspected, largely because of extra room in late years. • Largest effect is in late reserves- makes sense since this was a critical metric. • A 20% across the board reduction in exploratory Ps results in Reserves violations 2007-2009, minor early production violations. • The 20% reduction case can be reoptimized to honor all the constraints SPE 90395

30. Ps Sensitivity Conclusions • Reoptimized solution moves away from DW and Int’l, and towards the mid-continent. • If we use the original PS with the 20% reduction selections, there is little loss of performance, but we do exceed capital constraints in 2009. Reserve performance is better. • Overall, loss of Value is more severe than loss of needed performance. SPE 90395

31. Some General Data Considerations • In PM, frequency is usually more important than amplitude. In ranking and hurdling, amplitude is more important than frequency. • Beware the specific case. The general case is often more telling, and more subject to common sense QC. • Hurdles cause bias. • Any decision is valid. Manipulating data to justify a decision already made is not. • A vast amount of our “experience” has to do with generated data, not with actual data. • Beware unchecked internal limits- i.e. limiting only by internal cash flow- multiplies the effects of bias. SPE 90395

32. Finally • Bottom Line: • Shift from data paralysis to understanding the impact of data shortcomings. • Understand what does and doesn’t depend upon improving the data. • Be an analyst, not merely a compiler. SPE 90395