Background

1. T I M E OTHER HEALTHCARE DECISIONS Healthcare intervention reimbursement decisions Collaboration axis Explicit data needs Decision Clinical or policy decision Extrinsic value • Define explicit needs of decisionmakers • Establish reasonable requirements Decisionmakers Intrinsic value of treatment Value Matrix Multi Criteria Decision Analysis (MCDA) Decisionmaking body assessment Application axis Retrospective Application axis Prospective Quality of evidence Quality Matrix Criteria of quality International standards Decisionmaking body requirements Synthesized evidence Key info for each component of Value Matrix Fully referenced EVIDEM framework • Validate process in various jurisdictions • Explore context of past decisions • Generate data on quality of evidence • Adapt framework to existing decisionmaking processes • Basis to develop extrinsic value components Assessing & structuring evidence Investigators & experts Evidence Full text source documents (published, registries, proprietary) Development of evidence Researchers & manufacturers Collaboration axis Research planning • Planning tool for researchers and intervention developers to meet explicit needs • Develop methodology to generate data tailored to critical data needs 100% Proven safe and efficacious intervention to cure endemic severe disease resulting in major healthcare savings (e.g. malaria in Africa) 75% J I B F D G H C A 50% E 25% Intervention for a rare disease with minimal improvement in efficacy and major safety issues, resulting in major increases in healthcare spending 0% Interpretation of the VM scale (% maximum score) Decisionmaking process Scientificjudgment Valuejudgment Intrinsic value of product Extrinsic value of product Quality of of evidence Improvement over existing treatment None Lower Major Higher Validity of efficacy trial Low Small High Large Type of population treated (equity) Low priority? High priority? Number of patients Etc… Efficacy Etc… Vulnerable vs productive patients • Based on international standards • Comprehensive analyses can be systematized • Not highly dependent on evaluator perspective • Similar value system • Requires value judgment • Dependant on evaluator perspective • No shared value system • Dependant on region/institution priorities Evidence and Value: Impact on DEcisionMaking (EVIDEM ) – applying multicriteria decision analysis (MCDA) to support healthcare decisionmaking Mireille M Goetghebeur PhD,1 Monika Wagner PhD,1 Hanane Khoury PhD,1 Randy Levitt PhD,1 Lonny J Erickson PhD,1,2 Donna Rindress PhD1 1BioMedCom Consultants inc, Montreal, Quebec Canada 2Centre Hospitalier Universitaire de Montreal – McGill University Hospital center (CHUM-MUHC) Technology Assessment Unit, Montreal Québec Canada • Background • Healthcare decisionmaking is a complex process that requires simultaneous integration of numerous disparate types of information.1 • Population based and individual decisionmaking can be divided into two steps: scientific judgment to evaluate the quality of evidence and value judgment about the healthcare intervention.2,3 • There is a need, nationally and internationally, for transparent access to evidence and values on which healthcare decisions are made.4-11 • Multi-Criteria Decision Analysis (MCDA) is an established method widely used in numerous disciplines. It structures the decisionmaking process by breaking down the problem into the set of criteria which are expected to impact the value of an option.1,12 It has been applied within several areas of healthcare decisionmaking13-16 and presents a promising approach to reimbursement decisionmaking. • We hypothesized that healthcare decisionmaking could be facilitated by structuring the evidence and value judgments on which it is based into a practical and transparent architecture and that transparency would enhance understanding and implementation of healthcare decisions. • MCDA Quality Matrix • Assess the quality of all types of evidence for a healthcare intervention; • 12 QM components (rows) identified from literature review and requirements from more than 20 decisionmaking bodies worldwide17 organized by field of research; • 3 QM criteria of quality (columns) relating to: a) scientific quality (relevance and validity of evidence available); b) quality of reporting (completeness and consistency of evidence in high level documents and in individual studies), and c) adherence to decisionmaking body requirements; • Questions and instruments for QM cells derived from international scientific standards (e.g., CONSORT, CHEC, STROBE, Siegel et al, Mauskopf et al).19-23 • 4-step assessment: • Literature review for all types of evidence for the healthcare intervention; • Analysis of available evidence (public and manufacturer); • Analysis of requirements of decisionmaking body to which manufacturer dossier is submitted; and • Completing instruments, scoring and providing rationale for each QM cell. • Potential applications & developments • EVIDEM supports decisionmaking by structuring, segregating and providing transparent access to data, and by allowing communication of value judgments among stakeholders. • It can be applied retrospectively to generate data on quality of evidence or on past decisions and prospectively to integrate tools into existing decisionmaking processes and explore extrinsic components (Application axis). • On the collaboration axis, EVIDEM provides a practical framework to facilitate communication between those who generate data and those who need it to make decisions, facilitating future healthcare decisionmaking. • It can be used for any type of healthcare intervention, and by policy or clinical decisionmakers. • Future developments include collaborative studies and iterative processes to explore the value of EVIDEM in context as well as development of an interactive electronic platform integrating evidence and value for each healthcare intervention. • The expected outcome of a systematized and shareable approach for data access and value assessment is to optimize resources, decisions and health. • MCDA Value Matrix • Assesses the intrinsic value of a healthcare intervention from a specific perspective; it is applicable by decisionmakers at micro (patients, clinicians), meso (payers, institutions) and macro (healthcare policymakers) levels. • Components defining intrinsic value were identified through literature review, review of explicit criteria used by decisionmaking bodies, and also criteria used to prioritize technology assessment in numerous jurisdictions globally.17,18 • 15 components were selected to fulfill MCDA methodological criteria of completeness, redundancy, operationality & mutual independence1,12 and grouped into 4 clusters. • 2-step assessment: • Weighting of VM components from the societal perspective, independent of scoring healthcare intervention; and • Scoring of healthcare intervention using scale anchors & scoring guidelines combined with access to synthesized data (prepared using standardized methodology), quality of evidence scores & rationales (QM) and full text sources. • Synthesized evidence in the VM is a summarized HTA report tailored to decisionmakers questions; it is generated on a collaborative and on-going basis to provide timely answers • Objectives • Breakdown components of healthcare decisionmaking into practical tools that structure and quantify assessment of healthcare interventions • Build an architecture to provide timely access to transparent multiple layers of the components of decisionmaking • Ultimately, optimize healthcare and health through best use of healthcare interventions • Conceptual framework and architecture • A conceptual framework was developed that segregated components of decisionmaking into quality of evidence and two types of value judgment; intrinsic value of the healthcare intervention and extrinsic or system-related value. • It was hypothesized that segregating these concepts would make reasoning more explicit, increase transparency and facilitate complex healthcare decisionmaking. • Quantifiable components that shared commonly agreed direction of scoring were organized into matrices. • A practical iceberg architecture was designed to support decisionmaking using an MCDA approach to structure intrinsic value components (Value Matrix - VM) while providing transparent access to evidence (synthesized and full text) and quality of evidence (Quality Matrix - QM). • The architecture was built to facilitate communication between data producers and data users • Integrated procedures were developed to ensure efficient use of resources as this architecture is developed & applied to healthcare intervention assessment. • Proof of concept: Pilot study in Canadian context • The feasibility and value of EVIDEM was assessed by applying it to historical cases: 10 medicines (A-J) were assessed (cardiovascular disease, endocrinology, infectious disease, neurology, oncology, ophthalmology) using data from literature review and manufacturer dossiers submitted to the Canadian Common Drug Review and Québec Conseil du Médicament. QM scoring was performed by EVIDEM investigators. VM weights, scores and feedback on process were provided by the Canadian Value Panel, composed of representative stakeholders from across Canada (decisionmakers, specialists, generalists, nurses, pharmacists, health economists/epidemiologists). • Was the approach feasible? An algorithm was developed to operationalize each cell of the matrices and this was applicable to therapeutic areas and jurisdictions covered • Was it practical? It took 30 min on average to apply VM by stakeholders and about 250 hrs to build the structured package of fully traceable information • Value of EVIDEM according to panelists? structuring evidence, assessing strengths and weaknesses systematically, sharing between stakeholders – should not be used as a formulaic approach • Limitationsof pilot? Limited access to the underlying source data (e-icebergs in development) and extrinsic components to be explored • Acknowledgements • We wish to acknowledge the contributions of the Canadian Value Panel: Jean-François Bussières BPharm MSc MBA FCSHP, Director, Pharmaceutical Practice Research Unit, Department of Pharmacy, CHU Sainte-Justine Research Center; Benoit Cossette BPharm MSc, Pharmacist and Coordinator, Drug Disease Management Program, Fleurimont Hospital, Sherbrooke University Hospital Centre; Doug Coyle PhD, Associate Professor, Faculty of Medicine, Epidemiology and Community Medicine, University of Ottawa; Cheri Deal MD PhD FRCPC, Associate Professor and Program Director, Endocrine Service, CHU Sainte-Justine Research Center, University of Montréal; Roland Grad MD CM MSc CCFP FCFP, Associate Professor, Department of Family Medicine, McGill University; Christine Lee MD, FRCPC, Assistant Professor and Director, Microbiology Residency Program, Department of Pathology and Molecular Medicine, McMaster University; Mitchell Levine MD FRCPC FISPE, Professor and Director, Centre for Evaluation of Medicines, Department of Clinical Epidemiology & Biostatistics, Department of Medicine, McMaster University; Diane Lowden RN MSc (A), Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University; G.B.John Mancini MD FRCP, Professor and Program Director, Continuing Medical Education, Division of Cardiology, Department of Medicine, University of British Columbia; Paul Oh MD FRCPC, Medical Director of the Cardiac Rehab and Secondary Prevention Program, Department of Medicine, Clinical Pharmacology and Toxicology, University of Toronto; Genevieve Tousignant N MScN, Clinical Nurse Specialist, Montreal Neurological Institute & Hospital, McGill University; Wendy Ungar PhD, Associate Professor, Department of Health Policy, Management & Evaluation, University of Toronto, and Senior Scientist, Child Health Evaluative Sciences, The Hospital for Sick Children; Marie-Claude Vanier BPharm MSc, Associate Clinical Professor, Faculty of Pharmacy, University of Montréal, Family Medicine Clinic, Cité de la Santé de Laval. • This study was made possible by an unrestricted research grant from Pfizer Canada. Value of tested medicines on the VM scale References 1. Baltussen R, Niessen L. Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Eff Resour Alloc. 2006;4:14. 2. Eddy DM. Clinical decision making: from theory to practice. Anatomy of a decision. JAMA. 1990;263(3):441-3. 3. Tunis SR. Reflections on science, judgment, and value in evidence-based decision making: a conversation with David Eddy. Health Aff (Millwood ). 2007;26(4):w500-w515. 4. Dhalla I, Laupacis A. Moving from opacity to transparency in pharmaceutical policy. CMAJ. 2008;178(4):428-31. 5. Establishing transparency to restore trust in clinical trials. Lancet Neurol. 2006;5(7):551. 6. Rovira J. Transparency of economic evaluations of health technologies. Pharmacoeconomics. 2008;26(3):181-3. 7. Menon D, Stafinski T, Stuart G. Access to drugs for cancer: Does where you live matter? Can J Public Health. 2005;96(6):454-8. 8. Morgan SG, McMahon M, Mitton C, Roughead E, Kirk R, Kanavos P, et al. Centralized drug review processes in Australia, Canada, New Zealand, and the United kingdom. Health Aff (Millwood ). 2006;25(2):337-47. 9. Anis AH, Guh D, Wang X. A dog's breakfast: prescription drug coverage varies widely across Canada. Med Care. 2001;39(4):315-26. 10. Gregoire JP, MacNeil P, Skilton K, Moisan J, Menon D, Jacobs P, et al. Inter-provincial variation in government drug formularies. Can J Public Health. 2001;92(4):307-12. 11. Teng F, Mitton C, Mackenzie J. Priority setting in the provincial health services authority: survey of key decision makers. BMC Health Serv Res. 2007;7:84. 12. National Economic Research Associates. Multi-criteria analysis manual. On Communities and Local Government website [updated 2000; cited 2007 Apr 11]. Available from: http://www.communities.gov.uk/pub/252/MulticriteriaanalysismanualPDF1380Kb_id1142252.pdf.13. Le Gales C, Moatti JP. Searching for consensus through multi-criteria decision analysis. Assessment of screening strategies for hemoglobinopathies in southeastern France. Int J Technol Assess Health Care. 1990;6(3):430-49. 14. Mussen F, Salek S, Walker S. A quantitative approach to benefit-risk assessment of medicines-part 1: the development of a new model using multi-criteria decision analysis; part 2: the practical application of a new model. Pharmacoepidemiol Drug Saf. 2007;S42-S46. 15. Baltussen R, Stolk E, Chisholm D, Aikins M. Towards a multi-criteria approach for priority setting: an application to Ghana. Health Econ. 2006;15(7):689-96. 16. Nobre FF, Trotta LT, Gomes LF. Multi-criteria decision making--an approach to setting priorities in health care. Stat Med. 1999;18(23):3345-54. 17. Biomedcom Consultants Inc. BioMedCom Interface database. On Biomedcom Consultants Inc website [updated 2007; cited 2007 Sep 9]. Available from: http://www.biomedcom.org/en/resources-BMC-MADB.html.18. Noorani HZ, Husereau DR, Boudreau R, Skidmore B. Priority setting for health technology assessments: a systematic review of current practical approaches. Int J Technol Assess Health Care. 2007;23(3):310-5. 19. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285(15):1987-91. 20. Evers S, Goossens M, de Vet H, van Tulder M, Ament A. Criteria list for assessment of methodological quality of economic evaluations: Consensus on Health Economic Criteria. Int J Technol Assess Health Care. 2005;21(2):240-5. 21. The STROBE group. The STROBE statement: checklist of essential items. On STROBE Statement website [updated 2005 Sep; cited 2006 Jun 15]. Available from: http://www.strobe-statement.org.22. Siegel JE, Weinstein MC, Russell LB, Gold MR. Recommendations for reporting cost-effectiveness analyses: Panel on cost-effectiveness in health and medicine. JAMA. 1996;276(16):1339-41. 23. Mauskopf JA, Sullivan SD, Annemans L. Principles of good practice for budget impact analysis. On ISPOR website [updated 2007; Available from: http://www.ispor.org/workpaper/budget_impact.asp. • www.evidem.org