Heterogeneity and Comparative Effectiveness: Implications for Studies, Reviews, and Policy

1. Heterogeneity and Comparative Effectiveness: Implications for Studies, Reviews, and Policy David Atkins, MD, MPH Health Services Research and Development Quality Enhancement Research Initiative Dept. of Veterans Affairs

2. Session Overview:How to Examine for Heterogeneity and Its Implications • Within an individual study • Within a systematic review or meta-analysis • In Policy Decisions • Conclusions

3. Heterogeneity of treatment effect • Non-random variation in direction or magnitude of individual treatment effects within a population • May be due to factors specific to patient, provider, environment, or treatment • Also referred to as clinical heterogeneity

4. “Clinical trials are good experiments but poor surveys.” – N. Longford • From Kravitz et al.. Milbank Q. 2004;82:661-687

5. Heterogeneity vs. Applicability • HTE – How great is variation in the treatment effect for individuals within the study population relative to the “average” effect in that population? • Applicability – Do the results for the study population apply to a specific patient, group or setting of interest (outside of the study)?

6. “Clinical trials are good experiments but poor surveys.” – N. Longford • From Kravitz et al.. Milbank Q. 2004;82:661-687

7. Relevance in Policy Debates • Concern that “average” effects don’t apply to individual patients • “Cookbook medicine” • Decisions based on CER that ignore heterogeneity might deny clinicians ability to individualize treatment decisions

8. Is alendronate (Fosamax) better than calcium and vitamin D to prevent fracture? • What factors might cause heterogeneity in the net benefits of alendronate vs. calcium/vitamin D? • AUDIENCE RESPONSE

9. Sources of Heterogeneity in Net Benefits of Treatment • Responsiveness to treatment: • E.g., Calcium intake, vitamin D status • Vulnerability to harms • ? Risks from bisphosphonates • Baseline risk in absence of treatment • Age, health status, bone density • Values attached to outcomes • Convenience of regimens

10. Scenario • 63 year old woman with diabetes and ischemic heart disease, evaluated for cardiac procedure to address increasing exertional chest pain despite optimal medical therapy. She has two-vessel disease with preserved LV function. She is pondering whether to have PCA with stent or bypass.

11. Question • Is bypass surgery or percutaneous coronary intervention (PCI) superior for treating coronary artery disease? • Does the answer depend on patient or intervention factors (i.e. is there heterogeneity of the treatment effects )?