Economic Evaluations – What they are and what they are not

1. Economic Evaluations –What they are and what they are not Moira NapperHealth Economics Research UnitUniversity of Aberdeen

2. Issues Relating to the Health Economic Evaluation Literature • Methodological standards • Quality of reporting • Value-added information sources • Formulation of search strategies

3. Purpose of Economic Evaluation Studies • To ensure that the benefits from health care programmes implemented are greater than the the opportunity cost of such programmes by addressing questions of • allocative efficiency or • technical efficiency

4. Economic Evaluation Studies A FULL economic evaluation compares BOTH the • costs AND consequences of • two or more interventions Requires • identification • measurement • valuation of both costs and benefits

5. Economic Evaluation Methodologies • Cost-benefit analysis (CBA) • Cost-utility analysis (CUA) • Cost-effectiveness analysis (CEA) • Cost-minimisation analysis (CMA)

6. Economic Evaluation Methodologies

7. Partial Economic Evaluation Studies • cost comparison/cost analysis study • cost of illness study • cost outcome description • cost description

8. Mislabelling of Economic Studies Zarnke, KB; Levine, MAH; O’Brien, BJ. Cost-benefit analyses in the health-care literature: don’t judge a study by its label. Journal of Clinical Epidemiology 1997;50:813-822 Of a set of economic studies labelled CBA: • 68% did not use defined CBA methodology • 53% were cost comparisons only

9. Quality of Health Economic Evaluation Studies • Poor study design • poor data sources • failure to define clearly the economic evaluation method used • use of an inappropriate method if economic evaluation to address the research question • omission of sensitivity analysis to test robustness of modelling

10. Quality of Health Economic Evaluation Studies • Inadequate reporting • lack of transparency by authors • journal editorial policy surrounding economic submissions • Publication bias • publication bias in the clinical literature • motivation for conducting an economic evaluation

11. Guidelines for Economic Evaluation • Conduct • Pharmaceutical Reimbursement e.g. Australian “fourth hurdle” • Methodology e.g. NICE, Washington Panel* • Reporting • Drummond MF; Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. BMJ 1996;313:275-283 • Appraisal • NHS Economic Evaluations Database (NHS EED) guidelines for structured abstracts† • BMJ Guidelines *Weinstein MC. et al. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA 1996;276:1253-58 †Improving access to cost-effectiveness information for health care decision-making: the NHS Economic Evaluation Database. NHSCRD Report no.6 (2nd ed). York. NHS Centre for Reviews

12. Searching Bibliographic Databases for Economic Evaluations • Indexing policy • Availability of controlled vocabulary • Specificity of indexing terms • Indexing accuracy • Clarity of description • Accurate labelling • consistency

13. Indexing - MeSH Economics Costs and Cost Analysis Cost Allocation Cost-Benefit analysis Cost Control (+1) Cost of Illness Cost Sharing (+2) Health Care Costs Direct Service Costs Drug Costs Employer Health Costs Hospital Costs Health expenditures (+1)

14. Indexing - EMTREE Health Economics Economic Evaluation Cost Benefit Analysis Cost Control Cost Effectiveness Analysis Cost Minimization Analysis Cost of Illness Cost Utility Analysis Fee (+5) Health Care Cost Drug Cost Health Care Financing Hospital Cost (+3) Health Insurance (+8) Pharmacoeconomics(+5)

15. Value-added Bibliographic Sources of Health Economic Evaluations Databases • NHS Economic Evaluation Database (NHS EED) • Health Economic Evaluations Database (HEED) • EURONHEED • CEA Registry (formerly Harvard CUA Database) • Journals • Evidence Based Health Care • ACP Journal Club

16. NHS Economic Evaluation DatabaseNHS EED NHS Centre for Reviews & DisseminationUniversity of York, UK • Economic evaluations • critical appraisal detailed structured abstract • Other types of health economic analysis with bibliographic details • MeSH searching • 1995-

17. Health Economic Evaluations Database HEED Office of Health Economics, UK & International Federation of Pharmaceutical Manufacturers Associations (OHE-IFPMA).www.ohe-heed.com/ • Structured format • Bibliographic details • Study definitions • Analysis and results • Keywords • Expert review • 1992 -

18. The CEA Registry: Database of Cost-Utility Analysis Studies Harvard Center for Risk Analysis www.hsph.harvard.edu/cearegistry/ • Reference list of studies that have used costs per QALY to measure health benefits 1976 - 2001 • Comprehensive league table of Cost-QALY Ratios 1976-2001 • League table of cost-QALY ratios which adhere to the Washington Panel 1976-1997 • Catalogue of Preference Scores used for QALYs • Audit checklist

19. European Network of Health Economics Evaluation Databases (EURONHEED) Project • Launched 2003 - 3 year development • Country-specific databases set to be available in 2005 • 7 regional databases covering 17 European countries • 2 established viz. NHS EED (UK) and CODECS (France) • 5 to be created viz. Germany, The Netherlands, Italy,Spain and Sweden

20. Health Economic Evaluation Studies Summary Points • Strict definition criteria • Quality variation • Methodology • Reporting • Differences in bibliographic database indexing • Specificity and • Scope • Use of value-added sources

21. National Information Center on Health Services Research & Health Care Technology (NLM/NIH) • Etext of Health Technology Assessment (HTA) Information Resources (Chapter 11) www.nlm.nih.gov/nichsr/ehta/ehta.html • Health Economics Information Resources: a self-study course www.nlm.nih.gov/nichsr/edu/healthecon/