NICE Technology Appraisal: Methods & Practice

1. Technology Appraisal of Medical Devices at NICE – Methods and Practice Mark Sculpher Professor of Health Economics Centre for Health Economics University of York, UK

2. Outline • Policy context of NICE • NICE methods • Are devices different? • The role of randomised trials • When do we have sufficient evidence? • Impact of NICE guidance

3. Background • Brief overview of NICE • NICE’s decisions on devices • NICE’s methods: requirement for decision making • Are devices different? • The impact of NICE guidance

4. The National Institute for Health and Clinical Excellence (NICE) • Following election of Labour government 1997 • Prolonged controversy about ‘post code prescribing’ in the UK National Health Service • Wish to ‘de-politicize’ decisions about which technologies to cover in NHS • Desire to use best available methods to address difficult questions • Focus on drugs but devices also included

5. The NICE process Selection Assessment Appraisal • Specific technologies • Lacking in transparency • Subject to some criteria • Independent group • Review plus model • Good methods • supported • Assess company • submissions • 6 months or more • Companies can also • provide unpublished • data • Multi-disciplinary • committees • Take information from • range of sources • Range of decisions • possible

6. NICE decisions overall Source: Raftery, BMJ 2006.

7. NICE and medical devices (1) Source: Raftery, BMJ 2006.

8. NICE and medical devices (2) Source: Raftery, BMJ 2006.

9. NICE and methods • 2004 guidance more prescriptive than most international guidelines • Based on a clear view about the use of economic evaluation for decision making • Included the concept of the Reference Case National Institute for Clinical Excellence (NICE). Guide to the Methods of Technology Appraisal. London: NICE, 2004.

10. The requirements of economic evaluation for NICE-decision making Generic measures of health; QALYs Objective function Decision problem Clarity about population; full specification of options Appropriate time horizon Evidence base Context Time over which options might differ Inclusion of all relevant evidence Relevant to specific decision maker(s) Uncertainty Quantify decision uncertainty; feed in research prioritisation

11. NICE’s preferred methodology – the Reference Case Source: National Institute for Clinical Excellence (NICE). Guide to the Methods of Technology Appraisal. London: NICE, 2004.

12. What is the appropriate framework for economic evaluation? • Systematic review • Meta-analysis • Mixed treatment comparisons • Differing endpoints and follow-up • Patient-level and summary data Evidence synthesis • Structure reflecting disease • Incorporation of evidence on range • of parameters • Facilitates extrapolation and • separation of baseline and treatment • effects • Probabilistic methods Decision analysis

13. Are devices different?Decision problem • Need to include all relevant alternatives to the technology of interest • May include pharmaceuticals • May include sequences and other strategies (e.g. diagnostic) • Need to define relevant populations and sub-populations • May differ between jurisdictions No clear differences between devices and pharmaceuticals

14. Are devices different?Evidence base • Less likely to need trials for regulatory purposes • Does not mean should not be used for reimbursement • Typical ‘regulatory’ trials have limitations for economic evaluation • The evolution of devices over time • Not unique to devices • Has implications for evidence gathering • Need larger longitudinal studies, sub-groups on device types • Comparators may also be changing over time

15. Limitations of trials as a vehicle for decision making Trial limitations Inappropriate or partial comparisons More than one trial Partial measurement Unrepresentative practice Intermediate outcomes Limited follow-up No trials NICE Examples Temozolomide (recurrent malignant glioma) Drugs for Alzheimer’s Riluzole (resource use) Glycoproteins Beta interferon (MS) Implantable cardioverter defibrillators Liquid-based cytology

16. Type of parameter Baseline events risks Relative treatment effects Resource use/costs Quality of life/utilities Sources RCT control groups Observational RCTs preferred RCTs or observational RCTs or observational Synthesis issues Often long-term Often by sub-group Usually limited head- to-head data Formal synthesis rare Frequent need to map Formal synthesis rare Distinguishing types of parameters in models Cost-effectiveness models invariably draw evidence from multiple sources

17. Making trials more ‘naturalistic’The design continuum Comparators Placebo controlled All relevant comparators Measurement Few efficacy and safety endpoints Resource use, QoL Follow-up Shortest acceptable for registration Long-term follow-up Patients Tightly defined Reflective of full range of likely patients Practice Highly protocolised Reflective of routine practice

18. When is there sufficient evidence to reimburse? Implications of getting it wrong X Value of perfect information Decision uncertainty = • What is the probability • of the wrong decision? • Joint effect of uncertainty • in all inputs What are the implications of a wrong decision in terms of resources and health? • Sets an upper bound on the • value of further research • Can be calculated overall • and for individual parameters • Calculated per patient and • across a population of patients Sufficient evidence exists if it is not cost-effective to undertake further research

19. Evidence on impact of NICE decision on the NHS

20. Evidence on Orlistat for obesity Source: Sheldon et al. BMJ 2004;329:999.

21. Evidence on ICDs for arrhythmias Source: Sheldon et al. BMJ 2004;329:999.

22. What influences uptake? Source: Sheldon et al. BMJ 2004;329:999.

23. Conclusions • NICE part of an international trend towards greater use of economics in decision making • NICE is prescriptive about methods • Few differences between drugs and devices which affect appropriate methods • Impact of NICE guidance has been variable