The uptake of value of information methods Solutions found and challenges to come

1. The uptake of value of information methods Solutions found and challenges to come Alan Brennan Director of Operational Research ScHARR

2. Aim of this session • Discuss the roles of VoI analysis • Demonstrate recent technical progress • Growth in interest in and use of VoI methods • Discuss challenges for further uptake • Some recommendations • Personal Views !

3. What is Value of Information? • Given …. a choice between strategies, a decision rule for adoption of strategies, some uncertainty VoI analysis tells us… • How valuable more information would be to reduce uncertainty, to help us choose • Which uncertain parameters are most crucial • How valuable a sample of size n=10 would be compared to n=100, or n=1000 ….

4. Roles of VoI

5. Probabilistic Sensitivity Analysis Without VoI • C-E plane and CEAC • i.e. describes how uncertain we are

6. Probabilistic Sensitivity Analysis With VoI adds (1) • Which parameters or groups of parameters are important • i.e. the causes - why we are uncertain

7. Probabilistic Sensitivity Analysis With VoI adds (2) • Value of different samples and research design combinations • i.e. how best to resolve the uncertainty

8. Probabilistic Sensitivity Analysis - VoI difficulty myth • “It is complex” • “It takes a long time” • “Methods are not developed” • “People can’t understand it”

9. Probabilistic Sensitivity Analysis - VoI truth • If you have done a probabilistic sensitivity analysis i.e. • Then you are 90% to 95% there

10. Prioritising and Planning HTA • Recommendations from systematic review of the use of modelling in planning and prioritising trials (HTA 2003; Vol 7: number 23) • Results of Pilot Study show .. • Beneficial in refining research design and quantifying uncertainty • choice of timing and topic are vital • who should do the analysis is important

11. Industry Roles – Societal Perspective • Undertaking probabilistic sensitivity analysis and VoI to be ahead of re-imbursement authorities • VoI to identify likelihood of cost-effectiveness and priorities for further data collection • Very early Discovery phase work • Phase II results available – now design phase III • Phase III results available – what else do we need to make the economic case?

12. Industry Roles – Commercial Perspective • Approaches still developing (and commercial in confidence) • Define Net benefit not as λ * QALY – cost but rather as commercial sales / profit Or • model linking sales to likelihood of re-imbursement or extent of cost-effectiveness

13. Growth in UptakeA personal view • 1996 – Claxton and Posnett – “what are all these squiggles, it will never catch on” • 1999 – doing our review, realising the conceptual validity and practicability of VoI • 2000 – some applications but people are unsure on methods • 2001 – IHEA York / MDM San Diego 2 or 3 speeches (all UK)

14. Growth in UptakeA personal view • 2002 – CHEBS Focus fortnight with ourselves Karl Claxton and Tony Ades – methods sorted • Nice Appraisals beginning to use VoI • 2002 – MDM Baltimore -7 people UK and Canada – lots of interest • 2003 – MDM Chicago -15 to 20 people (US, Canada, Netherlands, UK) even more interest

15. Technical Problems Recently Solved

16. Technical Problems Recently Solved:EVPI • Correct method for EVPI = 2 level simulation EVPI = • 1 level simulation EVPI works if …. (a) the net benefit functions are linear functions of the -i for all of the decisions d and all of the possible values of the parameter set of interest i, and (b) if i and -i are independent. “

17. Technical Problems Recently Solved:EVSI • Correct Method for EVSI = 2 level simulation (Bayesian update given simulated collected data) • EVSI easy with conjugate distributions (i.e. retain same functional form when additional data is collected and synthesised) • Normal, Beta, Gamma, Lognormal • EVSI more complex without conjugacy

18. Technical Problems Recently Solved: Shortcuts • Complex models can be ‘emulated’ e.g. Gaussian Processes, which also offer quicker EVPI and EVSI for 1 variable calculation functionality • Laplace approximation can help shortcut to a 1 level simulation for EVSI calculations (poster) • The EVSI curve has a common but not universal shape – exponential of square root of n • Valuing additonal data in survival trials (poster)

19. Technical Problems Recently Solved: Software • EXCEL for models • R / Splus advantages • More sampling functionality • More optimisation functionality • Faster running times • Easier code • WinBUGS for complex posterior distributions – interlinkage with R and Splus

20. Key Challenges: Uptake for Sensitivity Analysis - AIDA • Awareness  Interest  Desire  Action • Publications / Conferences / Seminars • Training • York / Oxford Advanced Modelling Course • CHEBS characterising uncertainty and analysing outputs courses (2004) • Bristol WinBUGS course (2004) • Network of experts • Web resources

21. Key Challenges: Uptake in trial design • Engaging with traditional trial designers • Clinically significant difference is a proxy for the decision rule • Need to work together to compare results and approaches on pilot trials • particularly ones with Economic and Quality of life data to be collected

22. Key Challenges: Meeting Criticisms • How do you know uncertainty is properly characterised i.e. you might be uncertain about the uncertainty • You need to multiply by the number of people affected by the decision - How long does the technology / decision last – 2 years, 5 years, 10 years? An issue even without doing EVI • Information has value beyond the jurisdiction

23. Key Challenges: Technical • Correlation, Correlation, Correlation • Integrated VoI and evidence synthesis quickly • Reversible decisions • Deciding to wait – options pricing

24. Conclusions • Significant recent progress, interest and growth • Key challenges • Speeding up and teaching standard routines to do the calculations • Working through when exactly VoI is likely to be most valuable • Engaging sceptics to collaborate • Now is the time ….

25. Conclusions