Download
1 / 41
450 likes | 650 Views
Adaptive Designs: A Smarter Type of Clinical Trial. Martin Kimber , PhD, Tessella Ltd, February 2013. Tessella is an IT Consultancy. 33 years old, 250 staff, UK/US/NL Mostly custom software/analysis services for clients in science and engineering sectors
E N D
Adaptive Designs: A Smarter Type of Clinical Trial Martin Kimber, PhD, Tessella Ltd, February 2013
Tessella is an IT Consultancy • 33 years old, 250 staff, UK/US/NL • Mostly custom software/analysis services for clients in science and engineering sectors • ~ 30% in life sciences for pharmaceuticals and biotech
My personal professional background • Theoretical particle physicist to 2001 • Software engineering, major focus has been bioinformatics and statistics • Helping clients design and run adaptive clinical trials since 2007 in project management role • ~ 20 trials run so far • For all of these Tessella has partnered with a specialist stats firm called Berry Consultants • I like Bayesian statistics a lot
Adaptive Clinical Trials • Use pre-planned modification of some aspect of the trial as it is running, based on data collected during the trial, to • Improve ethics of trial • Improve scientific value of the trial • Improve efficiency of the trial whilst retaining the statistical validity of the results
This is not a statistics talk • For a medical audience – you need a contextual overview: • When do people use these adaptive designs? • How do they differ from conventional trials? • What are the implications? • For planning and preparation • For actually running a trial • For patients whom you might potentially recruit to such a trial … and I’m hoping you may find yourself asking… 4. Why aren’t more trials run this way?
Module 3 “Clinical Development” Learning Outcomes • Early studies in patients: dose-finding / proof of concept studies and their impact on drug development plan. • Clinical trial design (including legal, regulatory, ethical and practical aspects): international differences. • Principles and application of statistics in clinical trials. • Procedures for clinical trial data collection (paper and electronic) and data management (including validation processes) to ensure optimal quality data. • Key strategic issues in the clinical trial process, in terms of legislative requirements and Good Clinical Practice (GCP). • Legal and ethical provisions for protection of clinical trial subjects. • Various types of clinical studies in key therapeutic areas and the challenges in choosing the appropriate design and in conducting trials. • Collection, evaluation and reporting of adverse event data in clinical trials. • Various quality management issues in clinical trials. • Impact of emerging results on the drug development plan. • Evaluation of the outcome of drug development: final therapeutic profile / usage of a medicine. • Statistical issues in statistical report writing. • Evaluation and interpretation of clinical trial results. • Principles and practical application of critical appraisal.
1. When do people use these adaptive designs? What diseases? What stages of drug development? What research questions are being asked? What resources are involved? When are they not likely to be used?
Indications we have run adaptive trials in • Stroke (landmark early trial “ASTIN” 1999-2001) • Diabetes, Alzheimer’s, Migraine, Schizophrenia, Bi-polar disorder, Opiate-induced constipation, Rheumatoid Arthritis, Prostatitis, Bladder Pain, Oncology… • For many major pharma and some virtual/biotech firms • Berry Consultants also involved with academic trials and medical device trials • Across the industry, adaptive designs undoubtedly being used in many other indications too in drug development • BUT still only a small percentage of all late-phase trials currently being run are adaptive at all
Types of Adaptive Trial Design (include…) • Phase 1 – dose escalation (ascending cohort studies) • Phase 2 – single interim analysis for futility (e.g. 2a/b) – adaptive allocation dose response (many interims) – arm-dropping (single or multiple interims) – sub-population selection (ditto) • Phase 2/3 – seamless transition if condition met – often dropping most arms / sub-populations • Phase 3 – group sequential – sample size reassessment LEARN { } CONFIRM
What research questions? • Phase 1: find the maximum tolerated dose (oncology) • Traditional 3+3 is improved upon by CRM which models all data • Phase 2a: proof of concept – is there some efficacy? • Early stopping for futility saves everybody’s time & money • Phase 2b: dose ranging – which dose is optimum? • Study lots of doses, find maximum efficacy, minimum clinically significant efficacy, or 90% of maximum, or maximum utility • Find a sub-population that responds to new treatment (e.g. genetic signals for breast cancer) • Phase 3: confirm efficacy – but use simple adaptations to abandon a failing trial, or extend a potentially inconclusive one to avoid failure
Typical Adaptive Allocation Dose Finding Some extra work to enact a complex design time
What resources are involved? • Extra planning upfront • Statistics, Randomization and Drug Supply • Simulation strongly advised • Real-time or close to real time data collection • The primary endpoint is key – or a few endpoints • Don’t need to get all auxiliary data for interim analyses • Modern EDC systems likely to cope (but setting up the first such trial may require consultancy assistance!) • Data Monitoring Committee during recruitment • Not just a DSMB – need statistical analyses of efficacy • Predefined rules to apply the adaptations • Maintain study blind
When would adaptive designs not be good? • It’s mostly about time-to-information • Adaptive designs could be of little use if: • Recruitment is very fast – and/or subjects comparatively ‘cheap’ • OR it takes a very long time to get final endpoint data (unless we have a plausible longitudinal model to make projections – actually we often do and have successfully run ~18-month trials with 12-month endpoints, based on early results at 3-, 6-, 9- months etc) • Note that “we only have 3 doses” in phase 2 is not necessarily a reason not to use adaptive • Is there a real reason only to have 3, or is there an assumption that there is only sample size to have 3 under a fixed design?
2. How do they differ from conventional trials? Why should the statistics be harder to do? Is it more expensive to run a trial this way? Is there any ethical impact? Why does the FDA recommend doing simulations? Can a trial like this go wrong? If so, how?
Statistical complications • Multiple testing problem makes statistical testing harder • For any tests with multiple possible doses/populations/etc • Problem amplified by multiple assessments of the emerging data by successive interim analyses in an adaptive trial • Concern is inflating the Type-I error (false positives) • Eg measure a ‘blip’, follow the ‘blip’, not as significant as it looks • Actually, can make better use of the data (in terms of power to find a real signal) with techniques like dose-response modelling – smooth together adjacent doses • Naturally leads to a Bayesian framework, has other advantages like being able to incorporate some prior information if available • … but not just “simple” frequentist stats tests
Is it more expensive to run a trial this way? • Not much • Most trial costs the same (mid-size multi-centre phase 2) • More planning effort before the trial • Extra effort during trial to ensure early data is available correctly and processed to adapt • May be extra staff time and infrastructure/consultancy costs • Likely to be more noticeable for running a first adaptive trial with given set of vendors (unless very experienced) • You may save money by stopping early • At the drug development programme level it is fairly easy to argue that adaptive techniques make better returns
Is there any ethical impact? • More complicated to explain to patients – but informed consent shouldn’t require statistical expertise • Adaptations should be hidden from blinded participants (patients, doctors, CRAs, monitors, most sponsor staff) so not very different from a non-adaptive trial • Interim results restricted to small group of unblinded people, who should not be active investors in the pharma or competitors, or make decisions where there could be conflict of interest • Less clear when trial will stop recruiting • Any double-blinded randomized controlled clinical trial has the risk your patient might be on placebo
Why do simulations? • Recent FDA guidance encourages simulations for all trials • For adaptive trials: • A) the statistical properties may not be something you can calculate directly – simulation estimates operating characteristics (e.g. “4.2% of such trials generate a spurious success result when dose-response is in truth flat, under our assumptions of how variable the endpoint is”) • B) operational aspects, like recruitment rate (and how quickly data is gathered), can actually affect the efficiency of the trial’s adaptations and hence the statistical performance too • For all trials: • Using a realistic simulator helps the process of planning logistical details associated with recruitment and drug supply, and reduces the risk of problems later in the final analyses • E.g. the impact of missing data if many patients drop out • Or greater measurement variability
Can an adaptive trial go wrong? • Recruit too fast => little chance to adapt • (Some designs effectively reduce to a fixed trial if so) • Technical errors may affect the chosen randomization ratios, so can’t be undone at the final analyses • Logistical errors such as accidental unblinding could invalidate trial integrity (therefore careful SOPs needed) • Could blinded participants/observers try to game the algorithm in some way by timing of randomization or by attempting to infer something from incidental changes?
3. What are the implications? For planning and preparation Does it take longer to set up? Will the regulators approve the trial design?
Planning and preparation - Timing • Vendors need to be aligned to play correct role at correct time – more complex contracts, everyone needs to be ready as trial starts recruiting • For a phase 2 may well need extra 6 months – 1 year of simulation, planning logistics, etc, at least for first adaptive trial in particular indication/with particular set of vendors • We work with pharma companies who are successfully industrialising this • Short start-up time to trials with similar designs, similar stakeholders, related indications
Will the regulators approve the design? • Different regulators (FDA, EMA, etc) have different levels and styles of involvement • E.g. FDA may like to advise on how to design an adaptive phase 2 • Of course, they have a smoother ride if reviewer familiar with similar trials and techniques • Best not to reinvent the wheel from scratch – templates of standard adaptive designs will emerge – software is available (!) • Often a pharma company’s own regulatory department, or consultants who have never done adaptive, may take an exceedingly pessimistic view of “what FDA might say” • Revisions/comments should also not be overinterpreted because regulators never guarantee that one trial will be able to produce unambiguous results
3. What are the implications? For actually running a trial How is the integrity of the trial blind maintained? What value is achieved by adaptation? What happens to the overall timeline?
Integrity of the trial (blinded & randomized) • Sensible design never loses its randomness • Extra SOPs for who can know what and who makes decisions • Important to emphasize that designs are not ad-hoc (the adaptations should be preplanned) • Sometimes a more complex design (such as multiple interims and adaptive allocation) may be better to keep investigators fully unaware than a simple single interim • E.g. an inference that a proof-of-concept trial has passed its futility interim being interpreted to mean drug is OK; or trying to “work out” if an arm has been dropped and guessing why
Value of adaptation • Substantial savings if can stop early (for either reason) • Better information if can focus resources on dose/subgroup INTERESTING PART OF D-R END RESULT OF DYNAMIC ALLOCATION EIGHT DOSES VERSUS PLACEBO
Overall timeline? • Sometimes shorter (stop early) • More often much the same • Could be longer (if recruiting slowly, but results unclear) • Probably what matters is the whole phase 2 to phase 3 drug development programme time – for several candidates in therapeutic area • Those who focus on shaving as much time as possible on getting the drug to market must have a strong prior belief that the asset is efficacious • (Something like 50% of drugs fail phase 3)
3. What are the implications? For patients whom you might potentially recruit to such a trial Are they more likely to benefit directly? Are there different risks they may face because the trial is adaptive?
Patient benefits • Yes, more likely to be on an effective dose • Either because an ineffective dose won’t be trialled in as many patients (trial stopped before you got on it) • Or because an adaptive randomization scheme will avoid allocating to less good doses (ineffective, or bad side effects in some designs) • Still have a chance of placebo/existing standard of care, but the design can explore more potentially effective doses and avoid sticking many patients on too low a dose, for instance • However it would not be wise to try to game the randomization or timing of entry of your patients into the trial • Always some tension between science and individuals • Future patients benefit from smarter drug development
Patient risks • Not really different risks because of adaptive trial per se, assuming it is properly designed and honestly run • Most trials are using a similar (perhaps finer grained) set of doses and the same kinds of inclusion/exclusion criteria • Risk that trial is cancelled for futility during their treatment period (depending on protocol) • But if so that will only be because there is accumulated evidence that their treatment is worthless or even harmful • If a trial stops taking on new recruits for success, or transitions to a new phase, the protocol should have arrangements for the continued care of those already in the trial
4. Why aren’t more trials run this way? This all sounds challenging. Is the extra complexity really worth it? Is this storing up trouble when it comes to regulators? Are perceived barriers to adaptive trials being overcome?
Crystal ball: adaptive will increase slowly • Conservative pharmaceutical industry • Smarter holistic approach forced on them by patent cliff • Teams believe in their drugs – don’t want to kill them off! • Confusion about methods and regulators (often hearsay) • 10 years ago supporting technologies were not very widely available, but surveys of pharma companies now show these barriers are reduced • Reporting bias – don’t trumpet the “fast fail” to investors – and the success stories focus on the drug not the methods used to triage it and refine its dosage etc • But eventually successes will encourageall firms to follow…
