1 / 21

Identifying evidence for decision-analytic models

Identifying evidence for decision-analytic models. Suzy Paisley DoH Research Scientist in Evidence Synthesis Consensus Working Group on the Use of Evidence in Economic Decision Models : Establishing the Current Situation MRC HSRC Workshop 25 July 2005. DoH Award.

lynn
Download Presentation

Identifying evidence for decision-analytic models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Identifying evidence for decision-analytic models Suzy Paisley DoH Research Scientist in Evidence Synthesis Consensus Working Group on the Use of Evidence in Economic Decision Models : Establishing the Current Situation MRC HSRC Workshop 25 July 2005

  2. DoH Award To develop a systematic approach to searching for evidence to inform decision-analytic models • Characteristics and sources of evidence to inform parameter values • Use of evidence in informing other parts of the modelling process Consensus Working Group on the Use of Evidence in Economic Decision Models

  3. Today’s presentation • Why an issue • Current practice • Identification of evidence • Use of evidence • Implications of current practice Consensus Working Group on the Use of Evidence in Economic Decision Models

  4. Why an issue? • Environmental issues • Own experience • Undertaking HTA searches • Peer review of HTA reports Consensus Working Group on the Use of Evidence in Economic Decision Models

  5. ‘Typical’ HTA • Decision problem / research question • Syntheses of existing evidence • Systematic review of clinical-effectiveness • Decision-analytic model of cost-effectiveness Consensus Working Group on the Use of Evidence in Economic Decision Models

  6. P = colorectal cancer I = irinotecan C = any relevant O = survival #1 colorectal #2 irinotecan #3 rct #4 #1 and #2 and #3 Decision problemsearch question Consensus Working Group on the Use of Evidence in Economic Decision Models

  7. Modelling decision problems • Effectiveness • Costs • Resource use / activity • Health states • Utility values • Indirect comparators • Longer term outcomes • ‘Other’ interventions • Natural history • Epidemiology Consensus Working Group on the Use of Evidence in Economic Decision Models

  8. Current practice • Analysis of 27 technology assessment reports (TARs) • Review of HTA guidelines • Content analysis of modelling guidelines Consensus Working Group on the Use of Evidence in Economic Decision Models

  9. Current practice:Reports of models • Use evidence which cannot be identified systematically by PICO focused searches (21/27 (78%) assessments for NICE Paisley, 2001) • Report PICO focussed searches (4/21 (19%) reported additional ‘ad hoc’ searches Paisley, 2001) • Issues of transparency Consensus Working Group on the Use of Evidence in Economic Decision Models

  10. Types Effectiveness Costs Resource use / activity Health states Utility values Indirect comparators Longer term outcomes ‘Other’ interventions Natural history Epidemiology Sources ‘Published papers’ Routine data Reference sources Local / clinical / expert opinion Sponsor submissions Current practice:Reports of models Uses Parameter values Model structure Sensitivity analysis Validation / consistency / calibration Consensus Working Group on the Use of Evidence in Economic Decision Models

  11. Current practice:Methodological guidance • HTA Guidelines • Modelling guidelines Consensus Working Group on the Use of Evidence in Economic Decision Models

  12. Current practice:HTA guidelines (n=32) Identification of evidence: No mention 13 (41%) Systematic review approach 13 (41%) Broad approach 2 (6%) Approach unclear 2 (6%) Not available 2 (6%) Name data sources 21 (66%) Consensus Working Group on the Use of Evidence in Economic Decision Models

  13. Current practice:Modelling guidelines ISPOR Principles of Good Practice (Weinstein et al, 2003) • 1 explicit reference to systematic reviews of the literature • 41 implicit references to a process of seeking evidence • 30/41 (73%) references form statements within recommendations Consensus Working Group on the Use of Evidence in Economic Decision Models

  14. Current practice:Modelling guidelines Model structure 10 (33%) Data identification 7 (23%) Data modelling 7 (23%) Data incorporation 0 (0%) Model validation 6 (20%) Consensus Working Group on the Use of Evidence in Economic Decision Models

  15. Current practice:Conclusions • Reports of models • Report PICO focussed searches • Use a wide range of evidence from a wide range of sources throughout the modelling process • Modelling and HTA guidelines • Give limited explicit guidance • Give an implicit indication that range of sources should be used throughout the modelling process Consensus Working Group on the Use of Evidence in Economic Decision Models

  16. Effectiveness Costs Resource use / activity Health states Utility values Indirect comparators Longer term outcomes ‘Other’ interventions Natural history Epidemiology Defining search questions Defining the evidence base Transferable / ‘proxy’ data Grouping questions Implications for searching:Types of evidence Consensus Working Group on the Use of Evidence in Economic Decision Models

  17. ‘Published papers’ Routine data Reference sources Local / clinical / expert opinion Sponsor submissions Prioritising appropriate sources Implications for searching:Sources of evidence Consensus Working Group on the Use of Evidence in Economic Decision Models

  18. Parameter values Model structure Sensitivity analysis Validation / consistency / calibration Iterative nature of modelling process Matching retrieval of evidence to use of evidence Implications for searching:Uses of evidence Consensus Working Group on the Use of Evidence in Economic Decision Models

  19. Other issues • Aim of the search • What evidence? • What for? • How much? • Methods of selecting evidence • Alternative search methods • Reporting standards Consensus Working Group on the Use of Evidence in Economic Decision Models

  20. Conclusions The process of identifying evidence for models needs to take account of the following issues: • The range of types, sources and uses of evidence • The process of identifying relevant evidence bases and defining search questions • The iterative nature of model development • How much evidence is required • Reporting standards Consensus Working Group on the Use of Evidence in Economic Decision Models

  21. Conclusions Identifying evidence for models is different to identifying evidence for systematic reviews. Consensus Working Group on the Use of Evidence in Economic Decision Models

More Related