A misplaced role for cost effectiveness analysis in the economic evaluation of public health?

1. A misplaced role for cost effectiveness analysis in the economic evaluation of public health? David Cohen University of Glamorgan Funding from Welsh Assembly Government Wanless Health Economics Programme

2. Some background • WAG funds Wanless Health Economics Programme bringing together health economists with an interest in public health • Agree that first task should be to commission systematic review of economic evaluations of public health • LSE review shows only 2.5% of studies meeting inclusion criteria are cost benefit analyses • WAG funds further work from Glamorgan (Cohen), Bangor (Edwards), Aberdeen (Lubrooke) and Brunel (Fox-Rushby)

3. Evidence Based Medicine • Need for EBM recognised at least 20 years ago • Broad acceptance that the evidence base must also include economic evidence • Medical treatments are provided by NHS • (Principle) aim of treatment is to produce health • Key economic question addressed by NICE = how can NHS use its resources to maximise health output?

4. But….. • There are many ways of producing health that do not involve medicine (NHS) • housing • environment • road safety • occupational health and safety • food policies • smoking policies • health promotion • etc., etc, etc, etc. • These all produce health by reducing the incidence/severity of preventable illness/injury

5. What is health economics ? A couple of good definitions • “Health economics is the application of the discipline and tools of economics to the subject matter of health” (Culyer, 1981) • “The study of how people and organisations make use of scarce resources and the rewards achieved in terms of health” (Earl-Slater, 1999) An example of a bad definition • “Health economics is a discipline that analyses the economic aspects of the healthcare industry, using methods and theories from economics and medicine” (Kielhorn and Graf von der Schulenburg, 2000) • According to this definition health economics has no role to play in public health

6. Two very different types of efficiency • Allocative efficiency - Should we do it? • Technical: - How should we do it? Different techniques of economic evaluation are used to address these different economic questions. They produce very different economic evidence

7. Cost Benefit Analysis (CBA) • Concerned with allocative efficiency • “Should we allocate scarce resources to this intervention?” • CBA addresses this question by identifying all costs and benefits, measuring them and then expressing them in money values • If value of all benefits exceeds value of all the sacrifices (costs) then social welfare is increased and the programme should be introduced

8. Cost Effectiveness Analysis (CEA) • Concerned with technical efficiency • Compares alternative ways of achieving an outcome • Outcome measured in natural units e.g. • smokers who quit • life years saved • QALYs • “What is the technically most efficient way of …..?” • CEA results only of use within the area in question e.g. smoking cessation • CUA extends this to “what is the technically most efficient way of producing a QALY”

9. But CUA often (normally) results in ambiguous conclusions • Clear conclusion = new intervention dominates (lower cost greater effect) • Ambiguous conclusion = neither dominates New interventions are most commonly both more effective and more costly So what do you do? • Do a CBA (is better outcome worth the higher cost?) • Derive an Incremental Cost Effectiveness Ratio (ICER) and apply some rule of thumb

10. Evidence based medicine • Main benefit = health (so aim is to maximise health from available resources) • Therefore need evidence on the technical efficiency of interventions (hence NICE preference for CUA) • Most common result of CUA is non-dominance (ICER) • But whether we should incur the extra cost in order to achieve extra effect is an allocative efficiency issue • Tendency to address allocative efficiency with heuristics e.g. NICE says that an incremental cost / QALY > £30,000 is ‘too high a price to pay’ • Essentially, this works quite well for health care

11. NICE heuristic says that ICER of < £20,000 is “worth paying” • But how can this be determined without identifying the magnitude of the opportunity cost and who will bear it? • Example of herceptin for early stage breast cancer? • So might evidence on technical efficiency be leading to reduced allocative efficiency?

12. Evidence based public health • Health care is clearly not the only way to produce health • Public health “… involves mobilising local, regional, national and international resources to create conditions in which people can be healthy” (Allin et al, 2004) • Improving the conditions in which people can be healthy can achieve important non-health benefits in addition to health

13. Allocative v. technical efficiency • A CUA of a public health intervention (technical efficiency) would only include QALYs on the benefit side • If CUA shows high incremental cost/QALY, implication would be that resources should not be allocated • CBA on this same intervention which included non-health benefits might show it to pass the CB benefit test • Therefore a policy decisions taken on basis of CUA might not be optimal from a social welfare perspective

14. CBA of policy to reduce air pollution CUA CBA Costs BenefitsCosts Benefits £--- QALYs £--- £a Health £--- £--- £b Cleaner air £--- £--- Total benefit = £Z £--- £--- Total cost = £X £--- Total cost = £Y • If CUA produces ICER > £30k message = policy is not ‘cost effective’ • But if £Y< £Z then adoption of the policy produces welfare gain • £Y< £b then policy increases social welfare even without health benefits

15. So what’s the problem? • If % of PH interventions producing non-health benefits is high compared with health care interventions, then the PH evidence base should include a higher % of CBA studies than health care • But in a systematic review by the LSE (McDaid & Needle 2007) 73% (1235 of 1697) economic studies in PH (73%) were CEA/CUA • Only 43 (2.5%) were CBA • The rest were (mainly) cost consequences analyses • Aim of our exercise: to identify the non-health benefits in economic studies in LSE review which were not CEA/CUA and consider these with respect to the health benefits included in the CEA/CUA studies

16. Finding useful papers • Identified papers = 462 • Abstract not available =19 • Abstract read = 333 • Full paper appeared useful by could not be retrieved =52 • Full papers read = 58 • Of these CBA = 10 CCA = 26 Multi-method = 14 Others = 8

17. Examples of non-health benefits identified • Avoided damage to buildings • Increased agricultural output • Improved climate change • Lower crime rates • Increased productivity • Higher number graduating high school • Higher housewives output • Avoided pain and grief • Avoided loss of leisure time • Savings criminal justice system • Transportation savings • Reduction in property damage • Savings from professional services • Avoided special education

18. A (fairly predictable) conclusion • Many PH interventions produce non-health benefits and these can be significant • Therefore many PH interventions need to be assessed by CBA • But number of CBA studies in PH is tiny • So one conclusion = we need more CBAs in PH

19. Q: Why don’t we see more CBAs in PH A: It’s bloody difficult But saying we will do A because B is difficult can be exceedingly dangerous because the message from A can be very different from the message from B

20. A possible way forward • Identify those PH interventions which have the greatest need for a CBA • Suggest doing this by placing intervention on a continuum from solely to incidentally health focused - solely health focused (e.g. adding folic acid to flour) - mainly health focused (e.g. road safety) - partly health focused (e.g. drugs and alcohol) - incidentally health focused (e.g. macroeconomic policies)

21. But CUA of even a solely health focused intervention (e.g. folic acid) • would not include short term effects e.g. reassurance to women of childbearing age from the knowledge that consuming fortified food reduces risk of conceiving baby with NTD • Unlikely to include non-NHS resource savings e.g. special education plus the tangible and intangible benefits to parents of not having a disabled child. • CUA might lead to forgoing of welfare gains – but not a major problem because these will be relatively small • The further along the continuum, the greater the likelihood that decisions based on CUA results will not be socially optimal

22. Overall Message • Much PH produces non-health benefits • Choice of economic evaluation technique should begin by examining the anticipated ratio of non-health to health benefits • Higher the ratio, stronger the argument to evaluate by CBA • High ratio PH interventions which are assessed by CUA should be treated with caution