Adaptive designs for NIHR funded trials: A health economics perspective

1. Adaptive designs for NIHR funded trials: A health economics perspective Stavros Petrou 26th November 2010

2. Outline • Brief introduction to trial-based economic evaluation • Economic perspectives on adaptive designs driven primarily by clinical, statistical and regulatory considerations: - Eligibility criteria - Statistical analysis plan - Maintain study power - Others - Outcomes unrelated to efficacy - Group sequential methods • Adaptive designs driven primarily by economic considerations • Implications for research funders

3. Trial-based economic evaluation • Aim: to maximise health gain given scare resources • Method: compare cost and consequences of interventions • Economic evaluation: explicit criteria for making choices • Incremental cost-effectiveness ratio (ICER): (CT - CC)/ (ET - EC) • Mean net benefit = Rc.E - C

4. Selecting treatment options Source: Gray A, Clarke P, Wolstenholme J, Wordsworth S. Applied methods of cost-effectiveness analysis in health care. Oxford University Press, 2010.

5. Adaptation of study eligibility criteria • Examination of baseline information blinded to treatment assignment • Modification of eligibility criteria to increase numbers of patients with desired characteristics • May impair relationship between treatment effect and changed patient characteristic(s) →Estimate net benefits from observed costs and effects, construct regression model with a treatment variable and covariates for patient characteristic(s) →Magnitude and significance of coefficients on interaction terms provide estimates of cost-effectiveness by patient characteristic

6. Adaptations to maintain study power (I) • Blinded interim analyses of aggregate data • Event rate may be below initial assumption • Increase sample size/period of follow-up → Trialists interested in events/time to event → Unlikely to collect comprehensive data on final outcomes, e.g. mortality → Economists interested in all events → Need to link intermediate endpoints to the long- term outcomes of interest → Extrapolation into life expectancy and quality adjusted life expectancy using survival models

7. Adaptations to maintain study power (II) • Stratification of patients at baseline • Following interim analysis focus remainder of study on subgroups with greatest event rates, lowest variance • Subgroups no longer the focus may contain important economic information: → most trials underpowered on economic endpoints → collect follow-up event, resource use and HRQL data in refined sub-groups • Analyse cost-effectiveness data by sub-group avoiding the danger of spurious differences in treatment effects due to chance

8. Adaptations based on outcomes unrelated to efficacy • Dropping dose group(s) with unacceptable rates of adverse effects • Requirement that adverse effect is not composite of or directly related to efficacy outcome: → patients in dropped dose group(s) may contain important economic information → follow-up → clinical assessment of risk-benefit comparison may be opaque → utility measures could synthesise risks and benefits → need for sensitive and validated utility measures

9. Adaptations using group sequential methods • Alpha spending approaches to control for Type 1 errors • Compelling ethical and statistical evidence needed for terminating early: (i) Futility (ii) Demonstrated efficacy • Is there a case for applying group sequential design and analysis methods to economic endpoints? → Frequentist and Bayesian approaches available for estimating sample size requirements for economic endpoints → Very complex: Economists tend to focus on estimation and not hypothesis testing → Typically large variability in economic measures might dictate very large sample sizes, which may be neither financially nor ethically acceptable

10. Adaptations to statistical analysis plan • Prospective SAP generally remains unaltered • Limited changes may be acceptable • Analytical plan for economic evaluation usually separate • Is there a case for introducing a decision criterion for extrapolation modelling for cost-effectiveness? →TOBY trial: Total body hypothermia for neonatal encephalopathy: pCE = 0.69 at 18 months, 0.99 at 18 years → REFLUX Trial: Minimal access surgery amongst people with gastro-oesophageal reflux disease: pCE = 0.46 at 12 months, 0.74 over lifetime → When does it become ‘futile’ to extrapolate cost-effectiveness?

11. Other unblinded adaptive designs

12. Adaptive designs driven primarily by economic considerations

13. Value of Information Expected value of perfect information (EVPI) Equals net benefit from the best decision we could make, minus the net benefit of the decision made based on current information Perfect information is unattainable; EVPI is therefore the absolute maximum we should be willing to spend on research Expected value of sample information (EVSI) Represents expected value of conducting a study of a specific size Can take account of trial costs, opportunity cost of delaying adoption, cost of reversing initial decision, etc Can be calculated analytically or using simulation Relies on many assumptions

14. Application of VOI to adaptive trial design • Developed by Andrew Willan • Sample sizes not dependent on type I and II errors • Aims to maximise ENG, i.e. difference between cost of trial and EVSI • Single stage EVSI design: sample size driven by potential no of patients and expected values • Incorporates pilot data on INBs (means, variances, between-patient variances). Also requires no of beneficiaries; fixed, variable & analytical costs for trial. • Two stage EVSI design: requires further complex assumption on optimal proportion of patients in first stage.

15. Adaptive design of early ECV trial • Pilot study: 323 pregnant women presenting in breech position randomised to early v late ECV • CIHR funded trial required 730 women per arm to have 80% probability to reject Ho if treatments differed by 8% using two-sided type 1 error of 0.05 • Single stage EVSI: assumes b0=69, v0=3725, σ2=217,227, N=1,000,000, Cf=$498K,000, Cv=$1,600 and Ca=$2,000; n=345 • Two stage EVSI: assumes α0=0.45; n=290

16. Source: Willan A, Kowgier M, Determining optimal sample sizes for multi-stage randomized clinical trials using value of information methods. Clinical Trials 2008;5:289-300.

17. Implications for research funders • Few economic perspectives on adaptive trial designs • Adaptive designs driven primarily by clinical, statistical and regulatory considerations will have economic implications • Only one study has generated an adaptive design driven primarily by economic considerations • Need for prospective experiments and validation • Adaptive designs for complex interventions, public health interventions