AIFA EUROPEAN CONFERENCE ON CLINICAL RESEARCH FOR DECISION MAKING

1. AIFAEUROPEAN CONFERENCE ON CLINICAL RESEARCH FOR DECISION MAKING March 30, 2007 External validity of clinical trials and transferability of their results to medical practice Luigi Pagliaro, Professor of Medicine, University of Palermo, Palermo, Italy

2. External validity of clinical trials and transferability of their results to medical practice • External validity or applicability • Applicability: information needed • Problems of applicability • Help for evaluating applicability • Final remarks • References

3. External validity or applicability In an RCT: • Internal validity refers to the comparability between the intervention (I) and the control group (C), and to the care for minimizing the risk of bias I. group C. group Trial results • External validity refers to the availability of information to guide the applicability of the trial results to the target population in a specific setting (1) External validity, applicability Target population and setting

4. 1. External validity or applicability • There is consensus that RCTs with adequate internal validity most reliably assess the efficacy of clinical treatments (2) • The statistical efficiency of RCTs is maximized in • the megatrialsthat enroll very large numbers of pts • belonging to different categories to detect • “moderate but still worthwhile treatment effects” • on major outcomes (3) • A fundamental idea underlying the logic of the megatrials is that the direction (although not the • size) of a treatment effect is similar throughout • the different categories of pts (4)

5. 1. External validity or applicability “The modest benefit ascribed to many treatments in clinical trials can be misleading because modest average effects may reflect a mixture of substantial benefit for some, little benefit for many, and harm for a few” From 5

6. 1. External validity or applicability  Megatrials are the best method to answer the question: “can this treatment work”? (6)  RCT Individual pts ……But the applicability of the treatment to the practice must take into account the characteristics of the individualpatients (7)

7. External validity of clinical trials and transferability of their results to medical practice • External validity or applicability • Applicability: information needed • Problems of applicability • Help for evaluating applicability • Final remarks • References

8. 2 Applicability: information needed (from 1, modified)

9. 2 Applicability: information needed (from 1, modified)

10. 2 Applicability: information needed (from 1, modified) • Many RCTs don’t report the information needed to evaluate the applicability of their results to the • practice, and/or they exclude specific subgroups of • pts that could benefit from the treatment (8-11) • Lack of this information is one of the main • reasons for the underuse in practice of treatments • that were beneficial in trials (1), and/or of the use • of a treatment in the “wrong” patients,sometimes • with harmful consequences

11. External validity of clinical trials and transferability of their results to medical practice • External validity or applicability • 2. Applicability: information needed • 3.Problems of applicability • 4. Help for evaluating applicability • Final remarks • References

12. 3. Problems of applicability RCTs lacking information needed to apply the results in groups of patients; some examples: • Treatment for patients with hypertension and comorbidities • Peg-interferon in HCV-related cirrhosis without knowledge of esophagogastric varices • Tolvaptan for hyponatremia in cirrhosis • Warfarin for patients requiring caution

13. 3. Problems of applicability • Inappropriate use of trial results to “wrong” • patients and risk of harmful consequences; • some examples • Endarterectomy for asymptomatic carotid stenosis in centres and surgeons with lesser expertise than those participating in the trial • Spironolactone prescribed to pts with heart failure more vulnerable to hyperkalemia than those in the trial

14. 3. Problems of applicability Comorbidity in hypertensive pts in primary care (12) Comorbidities were present in 89-100% of the Fortin’s pts potentially eligible to each RCT

15. 3. Problems of applicability Lack of information Heathcote EJ & al. Peginterferon alfa-2° in patients with chronic hepatitis and cirrhosis. N Engl J Med 2000; 343: 1673-80 (13) • The paper does not report whether patients with esophago-gastric varices were included in the study • This information would have been of help for decision making: esophagogastric varices (prevalence about 40%in Child A cirrhosis, target of the study) suggest an advanced stage of disease (14), with a lower probability of interferon response (15)

16. 3. Problems of applicability Lack of information Schrier RW & al (SALT trials). Tolvaptan, a selective oral vasopressin V2-receptor antagonist, for hyponatremia. N Engl J Med 2006; 355: 2099-112 (16); letters: 2007; 356: 961-3 • The trial excluded cirrhotics with Child-Pugh score >10 • The information about these pts would have been of paramount relevance: at least 50% of cirrhotics who develop significant hyponatremia have Child-Pugh score >10 (17, 18) • Only 67 cirrhotics were treated with Tolvaptan • The trial cannot provide any information on the risk of adverse events of the drug in cirrhotics

17. 3. Problems of applicability Lack of information Schrier RW & al (SALT trials). Tolvaptan, a selective oral vasopressin V2-receptor antagonist, for hyponatremia. N Engl J Med 2006; 355: 2099-112 (16); letters: 2007; 356: 961-3 • The trial excluded cirrhotics with Child-Pugh score >10; only 67 cirrhotics received Tolvaptan • The information about these pts would have been of paramount relevance: at least 50% of cirrhotics who develop significant hyponatremia have Child-Pugh score >10 (17, 18) • The trial cannot provide any information on the risk of adverse events of the drug in cirrhotics

18. 3. Problems of applicability Lack of information Hart RG & al. Antithrombotic therapy to prevent stroke in pts with atrial fibrillation: a meta-analysis.Ann Intern Med 1999; 131: 492-501 (19) • Exclusion criteria of the RCTs are often ill-defined (eg: “medical disorders” [20]; “contraindications for Warfarin or aspirin therapy” [21]), and information on co-treatments for pts with comorbidities is generally omitted • Clinicians may be uncertain about the risk/benefit of Warfarin in a substantial number of pts: a reason for its underuse (30-60% [22])?

19. 3.1 External validity and applicability to subgroups and individuals Baseline risk Endoscopic sclerotherapy for prevention of first variceal bleeding in cirrhosis: only RCTs with bleeding rate > 40% in untreated controls show a significant benefit (data adjusted for the duration of follow up); data subsequently confirmed (2) Heterogeneity X2: p < 0.0001 • From: Pagliaro L & al. Ann Intern Med 1992; 117: 59-70 • D'Amico G, Hepatology 1995; 22: 332-54

20. 3. Problems of applicability Inappropriate use of RCT results • The ACAS (Asymptomatic Carotid Artery Surgery) trial of carotid endarterectomy vs medical treatment rejected 40% of applicants,subsequently barring those with adverse operative outcomes (1) • However: in USA, most carotid endarterectomies are • performed by surgical teams whose complication rates • would have rendered them ineligible for ACAS (1,23): results of the ACAS trial and of 8 published series

21. 3. Problems of applicability Inappropriate use of RCT results • The RALES trial (24): Spironolactone significantly reduced all-cause & cardiac mortality in pts with severe congestive heart failure (NYHA class III-IV) • Pts with serum creatinine concentration > 2.5 mg/dl were excluded • Mean age 65 years; females 27%; proportion of diabetics not reported • Mean doses of spironolactone 26 mg/day; pts co- treated with β-blockers 10% No hyperkalemia-related deaths in the trial; however, in the years following the publication of the RALES:

22. 3. Problems of applicability Inappropriate use of RCT results Prescriptions of Spironolactone per 1000 pts RALES • After the RALES publication, spironolactone prescriptions greatly increased, even for pts more vulnerable to hyperkalemia than those in the trial (eg for elderly, diabetics, and for pts co-treated with β-blockers), or for pts excluded from the trial (ie HF in NYHA class I and II; mild renal failure); higher doses were often given (25, 26) RALES Hyperkalemia-related deaths per 1000 pts

23. External validity of clinical trials and transferability of their results to medical practice • External validity or applicability • 2. Applicability: information needed • 3. Problems of applicability • 4. Help for evaluating applicability • Final remarks • References

24. 4.Help for evaluating applicability Interpretive medicine (27, 28) The drug X is effective in the disease X1 The drug X can work in my pts with the disease X1, provided that: They have the characteristics a, b, c….m Evidence from an unbiased RCT However, there is no evidence to support the drug X if they have the characteristics n, o, p…..z An approach requiring time and competence

25. 4.Help for evaluating applicability Guidelines Guidelines built on the global evidence of many RCTs may fill in the gaps of information of the individual RCTs 84 72 Individual RCTs 5 Pre-appraised summaries Guidelines % preferences of 302 GP (From 29)

26. 4.Help for evaluating applicability Subgroup analysis A subgroup analysis can be worthwhile when established a priori from pathophyisiologic principles in trial populations probably heterogeneous (30,31), eg: • Potential heterogeneity in risks of treatment or in risk without treatment, eg baseline risk • Potential heterogeneity of treatment effect related to pathophysiology, eg multiple pathologies underlying a clinical syndrome; or genetic variation • Underuse of treatment in clinical practice due to uncertainty about benefit, eg underuse in specific groups of pts such as in elderly From 31

27. 4.Help for evaluating applicability Subgroup analysis Pharmacogenomic polymorphism Conclusions. “VKORC1 haplotypes can be used to stratify patients into low-, intermediate-,and high- dose warfarin groups and may explain differences in dose requirements among pts of different ancestries”(32) {VKORC1: Vit K epoxide reductase complex 1}

28. 4.Help for evaluating applicability Subgroup analysis Pharmacogenomic polymorphism Conclusions: in pts with IBD, very high TPMT activity predicts treatment failure (33) {TPMT: Thiopurine methyltransferase}

29. Some final remarks 1 • External validity refers to the information needed to promote the applicability of trial results to clinical practice • Main issues of external validity are the characteristics of randomised pts, comorbidities and co-treatments, outcome measures, and setting • Information gaps due to inadequate external validity may lead to clinical decisions unsupported by adequate knowledge on the most appropriate applicability of the treatments

30. Some final remarks 2 • Overviews and guidelines gathering the global evidence of individual RCTs and analysis of predefined subgroups may aid the transferability of clinical research to practice • A more effective remedy would be the inclusion of the most relevant issues of external validity in the design and reports of the RCTs submitted for approval or publication

31. References • Rothwell PM. Lancet 2005; 365: 82-93 • Guyatt GH & al for the EBM Working Group. JAMA 1993; 270: 2958-60 • Collins R & al. In: Maynard A & Chalmers I Eds. Non-random reflections on Health Services Research. BMJ Publ Group, 1997, p.197-230 • Peto R. J Clin Epidemiol 1995; 48: 23-40 • Kravitz RL & al. Milbank Quarterly 2004; 82: 661-87 • Haynes B. BMJ 1999; 319: 652-3 • Rothwell PM & al. Lancet 2005; 365: 256-65 • Hejat A& al. Arch Intern Med 2002; 162: 1682-88 (HF) • Herland K & al. Respir Med 2005; 99: 11-19 (COPD) • Uijen AA & al. J Clin Epidemiol 2007; 60: 330-5 (Hypertension) • Van Spall HGC & al. JAMA 2007; 297: 1233-40 (Eligibility) • Fortin M, & al. Ann Fam Med 2006; 4: 104-108 • Heathcote EJ & al. N Engl J Med 2000; 343: 1673-80 • Pagliaro L & al. Portal hypertension in cirrhosis: natural history. In: • Portal hypertension. Pathophysiology and treatment, J Bosch & RJ • Groszmann Eds. Blackwell Scientific Publ, 1994: p.72-92 • 15. Poynard T & al Gastroenterology 2002; 122: 1303-13 • 16. Schrier RW & al. N Engl J Med 2006; 355: 2099-2112

32. References 17. Angeli P & al. Hepatology 2006; 44: 1535-42 18. Biggins SW & al. Hepatology 2005; 41: 32-9 19. Hart RG & al. Ann Intern Med 1999; 131: 492: 501 20. Ezekowitz MD & al. N Engl J Med 1992; 327: 1406-12 21. GullovAL & al. Arch Intern Med 1998; 158: 1913-21 22. Choudhry NK & al. BMJ 2006; 332: 141-3 23 Hartling L & al. Ann Intern Med 2005; 142: 1100-11 24. Pitt B & al (RALES). N Engl J Med 1999; 341: 709-17 25 Juurlink DN & al. (Ontario Study) N Engl J Med 2004; 351: 543-51 26. Bozkurt B & al. JACC 2003; 41: 211-4 27. Horton R. Stat Med 2000; 19: 3149-64 28. Horton R. Can Med Ass J (CMAJ) 1998; 158: 245-9 29.Guyatt GH & al. BMJ 2000; 320: 954-5 30. Feinstein AR. J Clin Epidemiol 1998; 51: 297-99 31. Rothwell PM, Lancet 2005; 365: 176-86 32. Rieder MJ & al. N Engl J Med 2005; 352: 2285-93 33. Ansari A& al. Aliment Pharmacol Ther 2002; 16: 1743-50

33. stop