Issues in Selection of Deltas in Non-Inferiority Trials : Acute Bacterial Meningitis and Hospital-Acquired Pneumonia

1. Issues in Selection of Deltas in Non-Inferiority Trials :Acute Bacterial Meningitis and Hospital-Acquired Pneumonia John H. Powers, M.D. Medical Officer Division of Special Pathogen and Immunologic Drug Products Center for Drug Evaluation and Research U.S. Food and Drug Administration

2. Introduction • Clinical perspective on delta • definition of delta and components • impact of deltas in clinical setting • Delta 1 issues in acute bacterial meningitis and HAP • data from pre-antibiotic and antibiotic eras • confounders in determining efficacy of control regimens • Delta 2 issues with acute bacterial meningitis and HAP • consequences of less effective therapy • practical issues in selecting delta

3. Clinical Trials • Purpose of clinical trials • Distinguish effects of drug from other influences • spontaneous change in course of disease • placebo effect • biased observations • difficult for clinicians to make judgments on drug efficacy/safety outside of setting of clinical trial • high spontaneous resolution rate in less serious diseases • confounding factors for lack of patient improvement in serious diseases • lack of direct comparison of safety of two drugs in similar patient population

4. Non-Inferiority Trials • Non-inferiority trials attempt to prove test drug is not inferior to control drug by some margin • cannot statistically prove two drugs are identical in efficacy • need some way to estimate the variability around the difference between two treatments • Non-inferiority margin (delta) = maximum degree of inferiority of test drug compared to control drug that trial will attempt to exclude statistically • specified prior to initiation of trial

5. -15% -8% -20% +20% Non-Inferiority Margins • After completion of trial: 1) calculate difference in point estimates of efficacy of test agent minus control agent 2) calculate 95% confidence interval around difference in point estimates • gives some idea of variability around the estimate in the differences 3) compare lower bound of 95% CI to pre-specified non-inferiority margin

6. Components of Delta • Delta 1 • conservative estimate of advantage of active control over placebo • data-based • Delta 2 • largest clinically acceptable difference between active control and experimental drug • judgement based on consequences to patients of treatment failure • overall delta for clinical trial smaller of the two values • if delta 1 is large, overall delta set by delta 2

7. Components of Delta - Delta 1 • Historically-based data • Do we really know what we think we know? • lack of data from pre-antibiotic era • change in resistance patterns and epidemiology of organisms • differing response rates in sub- populations • changes in practice of medicine • problems with defining patients with bacterial infection vs. non-bacterial/non-infectious causes • different definitions of success and failure in current trials compared to previous mortality-based trials

8. Components of Delta - Delta 2 • Judgement based “acceptable loss” relative to current therapy • ideal situation • smaller delta for more severe disease • less loss relative to current therapy given potential for greater overall morbidity/mortality • larger delta for less severe disease • greater loss relative to current therapy may not translate into as great a consequence for patients • BUT we don’t live in an ideal world • practicalities of performing clinical trials

9. Components of Delta - various diseases • Acute bacterial meningitis • D1 = magnitude of advantage over placebo well-known AND large • D2 = decision on “acceptable loss” • Hospital-acquired pneumonia • D1 = magnitude of advantage over placebo not as clear • D2 = decision on “acceptable loss” • Acute exacerbations of chronic bronchitis • D1 = advantage over placebo unclear (and small?) • D2 = decision on acceptable loss not as critical

10. Components of DeltaMeningitis and HAP • Delta 1 - important questions Q1: What is the magnitude of benefit of any antibiotic therapy over placebo? Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit? Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial?

11. Components of DeltaMeningitis and HAP • Delta 2 Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease? • Scientific considerations • consequences of treatment failure in various patient subsets with meningitis or HAP • Practical considerations • effect of changes in delta on sample size as efficacy rate changes

12. Historical Data - Meningitis • Acute bacterial meningitis highly lethal in pre-antibiotic era • meningococcal disease most common and occurred in previously healthy young people • overall mortality 70-90% without specific therapy • mortality decreased to 30% with introduction of antimeningococcal serum • Flexner S. J Exp Med 1913;17:553-76 • sulfanilamide treatment reduced mortality to 10% (9/11 patients survived in original series) • Schwenker F et al. JAMA 1937;108:1407-8

13. Historical Data - Meningitis • Problems with historical data • different endpoints in current trials • developmental, neurologic, audiologic sequelae as well as mortality • different epidemiology • pneumococcal meningitis most common now in U.S. • different populations • proportionately more older adults with meningitis since introduction of HIB vaccine • Schuchat A et al. N Engl J Med 1997;337:970-6.

14. Historical Data - HAP • Clinical entity of HAP not described in pre-antibiotic era • only 2 spontaneous cures out of 151 cases in military recruits in S. aureus outbreaks in 1918 • few reports of gram-negative pneumonias • How certain is diagnosis in these case reports? • No way to compare antibiotic therapy to placebo

15. Historical Data - HAP • Celis R. Chest 93;318-24.1988 • 30.5% (33/108) all-cause mortality with “appropriate” antibiotics • 91.6% (11/12) all-cause mortality with“inappropriate” antibiotics • Alvarez-Lerma et al. Intensive Care Med 1996;22:387-94. • 16.2% (36 /146)attributable mortality with “appropriate” antibiotics • all-cause mortality 34.9% (51/146) • 24.7% ( 46/284) attributable mortality with “inappropriate” antibiotics • all-cause mortality 32.4% (92/284)

16. Historical Data - HAP • Problems with historical data • Difficulty in clinical diagnosis of HAP • patients in study who do not have disease • Change in nosocomial organisms over time • changes in resistance patterns • Different outcomes in various patient populations • mechanically ventilated pts. Vs. others • Death attributable to pneumonia vs. all-cause mortality • Clinical endpoints other than mortality in current trials

17. Components of DeltaMeningitis • Delta 1 - important questions Q1: What is the magnitude of benefit of any antibiotic therapy over placebo? Appears as large as 60%-80% mortality benefit but magnitude of benefit on clinical parameters not as clear Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit? Yes and No Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial? Yes

18. Components of DeltaHAP • Delta 1 - important questions Q1: What is the magnitude of benefit of any antibiotic therapy over placebo? May be anywhere from 8.5%-60% depending on how and in whom it is measured. Unclear benefit on clinical parameters Q2: Is the benefit of antimicrobial therapy in current trials measured in the same way as in the original trials showing benefit? Yes and No Q3: Is the magnitude of benefit of therapy over placebo large enough that it should not affect the selection of the overall delta for a trial? Point for committee discussion

19. Components of DeltaMeningitis and HAP • Delta 2 Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease? • Scientific considerations • consequences of treatment failure in various patient subsets with HAP • Practical considerations • effect of changes in delta on sample size as efficacy rate changes

20. Consequences of Failure • Meningitis • clear mortality benefit of antibiotic therapy • morbidity is developmental, neurological and audiological sequelae • what is magnitude of benefit of antibiotics? • HAP • mortality • magnitude of benefit varies depending on how and in whom it is measured • morbidity • increased costs and hospital stay • effect on rate of clinical resolution?

21. Practical Issues • Effect of success rate and delta selection on sample size • Selection of a smaller delta in more severe diseases with relatively lower success rates would increase sample size • Is larger sample size practical given: 1) epidemiology of the disease 2) limitations of inclusion and exclusion criteria 3) inability to continue on randomized therapy in studies of severe disease

22. Clinical Trial Implications:Sample size per arm to achieve 80% power D

23. Epidemiology of Meningitis* *Based on 248 cases in 1995 from Schuchat et al. N Engl J Med. 1997;337:970-6.

24. Epidemiology of Meningitis • Case fatality rates and incidence vary by organism • H. influenzae lower case fatality rates than S. disease caused by S. pneumoniae • S. pneumoniae now more common overall • mortality rates in future trials may be higher than those in past given shift in epidemiology • Number of cases in U.S. declining since introduction of HIB vaccine • estimated 12,920 cases in 1986 • estimated 5,755 cases in 1995 • Schuchat et al. N Engl J Med 1997;337:970-76.

25. Epidemiology of HAP • Actual incidence of HAP unclear (not a reportable illness) • NNIS data estimates 250,000 cases/year in U.S. • uses clinical definition of HAP • estimated 1% of all patients entering hospital develop pneumonia • 15-18% of all hospital acquired infections • 2nd most common after UTI • most common infection in ICU setting • ICARE report. Am J Infect Control 1999;27:279-84.

26. Epidemiology • Estimated U.S. cases per year (1994) • acute otitis media 26,000,000 • acute sinusitis 23,000,000 • tonsillitis/pharyngitis 21,000,00 • pneumonia (community) 4,000,000 • hospital-acquired pneumonia 250,000 • acute bacterial meningitis<10,000 • acute bacterial endocarditis 10,000

27. Recent Trials • Practical Points • success rates in HAP trials in 50% - 70% range • much larger sample size with smaller delta • recent approvals with 20% delta based on 1992 guidance in all recent HAP trials • theoretically a new drug could be as much as 20% less effective than comparator • almost half of patients do not complete trial • must take into account when planning sample size

28. Clinical Trial Implications:Sample size per arm to achieve 80% power D

29. Components of DeltaMeningitis and HAP • Delta 2 Q: What is an “acceptable loss” of efficacy compared to accepted therapy in a serious disease? • serious nature of meningitis and HAP would seem to call for selection of smaller deltas • smaller deltas would result in larger sample size of clinical trials - is this practical? • balance with risk of accepting drugs which may be 20% less effective than currently approved therapy • could be success rate of 40% for new drug for HAP

30. The Balance • Risk to patients of accepting larger deltas, especially in more severe disease versus • Realities of performing clinical trials