Information Assessment – “Base Case” and Range

1. Information Assessment – “Base Case” and Range

2. Developing meaningful, reliable information ensures a comprehensive understanding of the decision. Meaningful, Reliable Information: • Knowing what’s important • Having it correct and explicit • Based on appropriate facts • Including uncertainty • What we know • The limits of what we know Key tools: • Information research • Encoding of judgment • De-biasing techniques • Influence diagrams • Sensitivity analysis Failure modes: • Neglecting to obtain important information • Ignoring uncertainty • Missing interdependencies • Focusing on what we know, not what’s important • Ignoring “intangibles” 4 DecisionQuality 2 5 6 1 2.04 • Deterministic Information Assessment

3. We will discuss how to assess information for a deterministic model and sensitivity analysis. • What information to assess • Base case assessment and “10/90” ranges • Difficulties in assessing information There are several specific biases to avoid when interviewing experts—these will be discussed later. 2.04 • Deterministic Information Assessment

4. With a decision perspective, we can focus on the information that affects the decision. • What is the frame for our decision-making? Does the information we seek help us understand the problem as framed? • How will the information we gather provide insight about our choice of an alternative? • Will the value of collecting additional information justify the costs of acquiring it? 2.04 • Deterministic Information Assessment

5. Price Annual Revenue Price per Therapy day Therapy Days Market Share Sales Unit per Therapy Day Treated Population Therapy Days per Compliant Patient Compliance Rate* Competitive Class Treated Population Market Share over Time Share / Price Events Market Share Total Treated PatientPopulation Product Profile Comp. Class Share over Time Post LOE Share Effect of Launch Delay Market Size Class Event AfflictedPopulation Diagnosis Rate over Time Treatment Rate over Time Presenting Rate over Time Key Prevalence/Incidence Rate over time Uncertainty TargetDemographicPopulation over time Calculation/Point Estimate % of pop. that has health care access Relevance Influence diagrams help identify what information is required. 2.04 • Deterministic Information Assessment

6. Internal and external sources are considered to build the information base for decision-making. Information Gathering • Customers • Surveys • Market Research • Experiments • Competitive Analysis • Pilot Plant • Internal Experts • External Experts • Industry Analysis • Financial Modeling ImprovedInformation Base 2.04 • Deterministic Information Assessment

7. A good information base can help “calibrate” your assessments of uncertainties. • There is nothing better to improve the quality of your assessments then having some data. • But, beware! Data is always about the past. • Uncertainty means that the future can be very different. 2.04 • Deterministic Information Assessment

8. Agenda • What information to assess • Base case assessment and “10/90” ranges • Difficulties in assessing information There are several specific biases to avoid when interviewing experts—these will be discussed later. 2.04 • Deterministic Information Assessment

9. Deterministic Comparisons A B C 1 Base Case A B C 2 3 Sensitivity Analysis Uncertainty Range 4 5 Our initial analysis uses a “base case” value and uncertainty range for each variable. Spreadsheet Model 2.04 • Deterministic Information Assessment

10. No war, no strike, no fire, no flood, no price control, no major equipment failure, no new competitor, …. Begin by assessing low and high values of each uncertain variable. Expert Assessor 2.04 • Deterministic Information Assessment

11. Low Base High .8 Probability of Being “In the Range” 0 .5 1.0 .1 .2 .3 .4 .6 .7 .8 .9 “Low” and “high” values are defined using 10 percent probabilities. Value for Sensitivity Analysis Definition Name There is only a 10 percent probability that the variable will be less than or equal to this value. There is only a 10 percent probability that the variable will be greater than this value. There is a 50 percent probability that the variable will be less than or equal to this value. Low High Base Case 10th percentile 90th percentile 50th percentile (median) 2.04 • Deterministic Information Assessment

12. Document each assessment for easy reference during model and data refinements. 2.04 • Deterministic Information Assessment

13. Agenda • What information to assess • Base case assessment and “10/90” ranges • Difficulties in assessing information There are several specific biases to avoid when interviewing experts—these will be discussed later. 2.04 • Deterministic Information Assessment

14. Three paradigms frequently need to be addressed to obtain decision-focused information. • “Fact” bias • Unquantifiable intangibles • Lack of “hard” data 2.04 • Deterministic Information Assessment

15. ? ? ? ? Those with a “fact bias” often collect detailed historical data but ignore the future. This year Past years Future years -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 +6 +7 +8 2.04 • Deterministic Information Assessment

16. Possible Future Scenarios Those with a decision focus seek to understand the current situation quickly and then explore future possibilities. This year Past years Future years -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 +1 +2 +3 +4 +5 +6 +7 +8 • Current Situational Analysis • Customer • Competitors • Company 2.04 • Deterministic Information Assessment

17. Important intangibles can and must be evaluated. • Ballpark estimates based on experience and judgment come from answering, “Could it be larger than ...?” or “Could it be as small as ...?” • We can ask, “What tangible evidence would we see of this intangible?” • We can also reverse the process and estimate how large the impact must be to change the decision. Conservative accountants who assign zero values to many intangible benefits prefer being precisely wrong to being vaguely right. —Kaplan, HBR 2.04 • Deterministic Information Assessment

18. Although we lack “hard” data, judging a key factor’s range of possibilities is better than ignoring it. Housing Starts (year 2012) High Case = ? Base Case = ? Low Case = ? 2.04 • Deterministic Information Assessment

19. Judge the quality of available information before plunging into evaluation. Meaningful, Reliable Information: 0% “Blissful ignorance” • Don’t know how much we know • Ignoring uncertainty • Ignoring “intangibles” 50% “Informed about uncertainty” • Know information gaps • Know what’s important • Quantified uncertainty • Interdependence not explored 100% “Knowledgeable and ready” • Information correct and explicit • Important gaps filled • Know limits of knowledge 4 Clear Values and Trade-offs 3 Meaningful, Reliable Information 2 Creative, Doable Alternatives 5 Logically Correct Reasoning DecisionQuality 0% 100% 1 Appropriate Frame 6 Commitment to Action 2.04 • Deterministic Information Assessment