Planning and Execution of Learn-Phase Clinical Trials

1. Planning and Execution of Learn-Phase Clinical Trials Parvin Fardipour

2. Introduction • The planning and execution of response adaptive randomization trials involves complexities beyond those encountered in the planning and execution of more traditional trial designs • These complexities involve: • Availability and flow of information required to support adaptive decision making • Implementation of changes in the randomization scheme • Composition and role of data monitoring committees (DMCs) • Who can see the data, when and to what effect? • Drug supply

3. Decision to Use a Response-Adaptive Randomization Design • While learn-phase trials often have multiple clinical and scientific objectives, the primary goal(s) of the trial are used to guide the adaptive randomization • An adaptive randomization design is most likely to be successfully implemented when the clinical team support the approach but their enthusiasm is based on realistic expectations • Unlikely to be successful when the design is being used in a late attempt to “rescue” the development program for a compound with inadequate efficacy or an undesirable side effect profile • It is difficult to design and implement a response-adaptive randomization design as quickly as a traditional “off the shelf” design • Additional planning, validation, and implementation time may be poorly received in settings in which the key objective is to launch a trial quickly, rather than to launch a more flexible and potentially informative design

4. PLANNING PHASE • Selection of Endpoints • Identification of efficacy and safety endpoints for the trial, as applicable to the specific situation • Several issues merit consideration • The endpoints that drive the adaptive allocation rule must be available in a timely manner to allow modification of the randomization probabilities in real time • Ideally, the primary endpoint used in a learn phase adaptive randomization trial would be identical to the endpoint likely to be used in a subsequent confirmatory trial

5. PLANNING PHASE • Trial Objectives • The statistical design of a learn phase trial begins by asking the question • What is it we are trying to learn in this trial? – research question • We must also decide how much certainty is needed when answering the research question(s)? • In a dose finding study, the learning objectives are defined in terms of understanding some aspect of the dose-response relationship • for example, • What is the minimum effective dose (MED)? • What is the maximum tolerated dose (MTD)? • Stopping rules may written in terms of having answered the key research question(s) with the targeted degree of certainty; for example: • Stop for futility if, for every dose, the probability of a clinically significant advantage over placebo is <0.1 • Stop for success if we have identified at least one dose where the probability of a clinically significant advantage over placebo is 0.95 and we have identified the MED with a probability of at least 0.6

6. PLANNING PHASE • Development of an Adaptive Design • The adaptive design generally utilizes • A dose-response model (e.g. NDLM, Logistic) • And • Longitudinal modeling to incorporate information from enrolled subjects that have not completed the protocol • Treatment arms in the study should also be discussed with special consideration to logistical issues • How many treatment arms are required and what is the rationale? • PK and PD modeling • Feasibility of doses • Enrolment profile in the study is crucial for understanding information accrual and how the adaptations will be made during the study • It is important to discuss with the clinical team about the constraints to consider in the design such as the timelines (e.g., patent expirations, competitor projects), budget (e.g., number of centers, patients), clinical (e.g., highest number of tablets taken at any one time), drug formulations, maximum number of subjects

7. PLANNING PHASE • Proposed adaptive designs are evaluated by extensive simulation to assess operating characteristic of design across various response pattern scenarios (e.g. flat dose response, sigmoid dose response, etc) • The adaptive design is implemented as a software package that runs quickly enough to allow thousands of simulations to be run • The implementation software is configured to read in the trial data contained in the response and current randomization files and to produce the details of the model fit, the predictive probabilities for each stopping criterion, and the revised randomization scheme

8. PLANNING PHASE • Documentation • Even thought we are discussing a learn trial, we work under the assumption that the trial will become part of a future regulatory dossier; thus, all documentation should be in place prior to initiation of the trial and in a format that will facilitate regulatory review • These documents define the foundation for the execution phase and should be approved by sponsor personnel, DMC if applicable and then signed off, and stored in a submission-ready format • Required documentation that needs to be retained includes: • trial protocol, the original randomization scheme, the DMC charter, the adaptive design simulation report, SAP, etc

9. PLANNING PHASE • Documentation • All external parties that are involved in the execution of the trial must be identified and appropriate confidentiality agreements, conflict of interest disclosures, and contracts must be produced and approved by relevant authorities • The flow of clinical data and other information during the trial needs to be planned and managed • How are the reports generated by the adaptive design and the ISC shared with the DMC? • How are the DMC’s recommendations communicated to the sponsor’s steering committee? • What are the formats of the input file to the adaptive design software and the output file that must be loaded into the sponsor’s randomization system (e.g., IVRS)? • Has the process of loading the adaptive randomization file into the sponsor’s randomization system been tested and validated?

10. PLANNING PHASE • Validation Process • The adaptive design software is validated by a combination of independent source code review, analysis of simulation results and “back-to-back testing” • Back-to-back testing is performed by independently creating a second version of the program (or crucial parts of the model) • This process helps ensure the detection of programming errors and similar problems. The full validation process is documented in the validation report for the adaptive design • All the reports produced by the ISC must be validated and a validation and quality control report is produced • The complete data flow required for trial execution must be tested using test data

11. PLANNING PHASE • Validation Process • To load the adaptive randomization file output into the randomization system, a thorough validation of all the components must be performed • The update to the randomization system must be synchronized on the database and drug supply system to ensure that the next subject will be allocated to the new scheme • It is advised that upload of the adaptive randomization file into the randomization system is automated to avoid manual errors

12. PLANNING PHASE • The Response File • The response file is an input to the adaptive trial design software • The response file contains randomization identifiers, measurement time points if necessary, and the measured endpoint values for each relevant patient visit reported to date • The Randomization File • The randomization file is one output from the adaptive trial design software • The randomization file contains a new randomization allocation with treatment assignments for subsequent patients (until replaced by the next update) • In our trial, there is normally a burn-in period of balanced treatment assignment to drive the adaptation • The randomization file is imported into the clinical database, as well as the randomization and drug supply systems

13. PLANNING PHASE • Independent Statistical Center (ISC) • Charged with developing and executing programs for the creation of tables, listings, and graphs • Execute the adaptive trial design software • The data are transferred to the ISC via a secure connection and at frequent enough intervals to support the adaptive algorithm and allow production of the interim reports required by the DMC and specified in the SAP. • The results from the adaptive design are typically the estimated dose-response curve for the key variable(s), the predicted probabilities of meeting critical criteria (e.g., being the minimal effective dose), and also the new randomization allocation schema

14. PLANNING PHASE • Development of Interim Reports • DMC will sometimes have responsibility to monitor the study adaptation process, as well as having more general oversight for safety within the trial • This may lead to different types of interim analyses that must be addressed in the DMC charter (e.g. weekly interim and full interim) • Different reports will be generated to support each type of interim analysis. The type of reports required for each interims is documented in the SAP

15. BUSINESS CASE • it is important that a business model be developed to facilitate quantitative comparison of design options • During the requirement phase prior to the start of the trial, a key responsibility of the team is to identify the alternative (traditional) design if an adaptive design is not utilized • The alternative design will be used for comparison with the proposed adaptive design so that the business case justifying the latter can be developed • During the design phase, the business model is developed and applied in conjunction with the assessment of operating characteristics for various designs (including, in particular, the traditional alternative)

16. BUSINESS CASE • The key elements of the business case are: • What is the probability of the trial meeting its objectives (e.g., establishing proof of concept or identifying the minimally effective dose)? • What is the probability of success in phase III (i.e., the probability of moving from phase III to registration)? • What is the time to market? • What is the development cost based on number of subjects and duration of the trial?

17. BUSINESS CASE • In addition to the simulation work providing estimates of the expected number of subjects and the expected duration of the trial, the development cost estimate requires assumptions about various incurred costs such as monitoring, data management, drug supply, cost per patients and other generic costs such as investigator’s meeting, etc • Besides the various direct costs, any indirect costs such as utilized resources should be included in the equation as well • As with the assessment of operating characteristics, the business case assessment must be made across a range of response pattern scenarios • The business case is presented to the study team, considering the various response scenarios and facilitating comparison of the design alternatives • Once the final design is selected, the business case justification is incorporated into the portfolio management strategy

18. Data Monitoring Charter (DMC) Charter • The DMC Charter defines the roles, responsibilities, and activities of the DMC, the independent statistical center (ISC), the sponsor’s steering committee, contract research organizations (CROs) and other participating entities and/or individuals during the conduct of the planned adaptive clinical trial • Additionally, the Charter describes the purpose and timing of DMC meetings • The Charter also outlines the procedures for ensuring confidentiality and appropriate communication between the DMC and sponsor and the expectations for the Open and Closed Meeting Summaries that will be prepared by the DMC • The intention is for the charter to contribute to preserving the integrity and validity of the trial

19. Data Monitoring Charter (DMC) Charter • The function and composition of the DMC, adjudication board and executive committee need to be described • There are certain elements of the DMC that need to be considered prior to composition of the DMC • For example for the learn phase, the DMC could be composed of internal sponsor’s members unless there are extraordinary safety issues that require it to be external to the sponsor or to have external DMC members • The structures, members, roles and responsibilities of all parties are clearly defined • For example, does the DMC take into account important safety data originating from other trials within the same program? • If the DMC recommendation impacts drug supply, the communication between the DMC and drug supply must be defined to ensure that the integrity of the trial is not impacted

20. Data Monitoring Charter (DMC) Charter • The key part of any DMC charter is to define in detail the communication process flow and any operational issues such as conflict of interests, quorum, meetings’ purpose, timings and decision processes • The charter must document what information will be provided to the DMC members and how the DMC communicate to the ISC if additional reports are requested • If an adjudication board is required for the DMC, then the roles and responsibility and information flow to the adjudication board must be documented in the charter

21. Organizational structure Sponsor Steering Committee Study Team Lead Members Blinded Endpoint Adjudication Committee Study Team Blinded Unblinded Adjudicated endpoints Reports Independent Statistical Center Queries and requests for additional reports Reports Clinical data Clinical Data DMC Liaison Queries and requests for additional reports Reports Recommendations Data Monitoring Committee

22. Adaptive Design Model Requests Stop Trial (efficacy, futility) Stop the Trial or an arm? Committee Agrees? Full Interim Analysis Stop Trial (safety) Full Interim Analysis Stop an arm (safety) Communication Flow Diagram – Example Trials continues YES - for stopping an arm, applies only if the decision is not to inform the sites YES YES Stop Randomization (an arm) Communicate to Steering Committee DMC convene a meeting YES - for stopping an arm, applies only if the decision is to inform the sites YES Communicate to Study Team YES Stop Randomization (trial or an arm) Communicate to Sites

23. ACCEPTANCE PHASE • Once all the programs are validated and quality controlled, programs are moved into the production area and the acceptance phase starts • During the acceptance phase, the reports are generated in blinded format • The goal is to ensure that the formats of the reports are acceptable to all reviewers and to add any additional reports that are requested by the DMC. • The acceptance phase is used to ensure that the system is functioning as designed • For the acceptance phase, live data can be used (normally around 5 subject’s data) • Once all reviewers agree with the results, the execution phase begins • It is critical that adaptive design software is executed during the acceptance phase to ensure that the results are accurate

24. EXECUTION PHASE • The execution phase begins once the acceptance phase is complete • The time-points for transferring data are defined in the DMC charter • Also, it is defined in the charter and/or SAP what reports are produced for what group (i.e., what DMC receives, what the adjudication board receives, etc.) • Data are transferred via secure connection to ISC • ISC produces reports and performs a quality control prior to releasing these reports to DMC • In general DMC reports should be produced with unblinded treatment displayed, however, appropriate data security must be in place • Hence, it is important that the reports are stored in secure location for DMC to review. • Patient listings should be limited and should be justified based on DMC purpose

25. EXECUTION PHASE • DMC meets according to the schedule that is defined in the charter and meeting minutes are noted and stored in a secure location • For adaptive design studies, there should be time allocated aside for DMC to deliberate when adaptation is executed • DMC follows the charter when it requires communicating its recommendation • The adaptive design software output consists of a revised set of randomization scheme, as well as predictive probabilities for key trial outcomes • The randomization file is forwarded to the person who is responsible for loading the new randomization probabilities into the interactive voice randomization system (IVRS), verifying the new randomization tables, and informing DMC that a new randomization table has been loaded

26. EXECUTION PHASE • Additional Reports Requested by the DMC • The chair of the DMC may ask the ISC to produce additional new reports and/or to prepare an interim report package at unscheduled time points • These requests must be validated and quality controlled prior to release to the DMC and also documented by the chair or designee in a secure location • It is understood that the adaptive design software must not be modified during the trial and hence any additional requests must not interfere with on-going execution of the adaptive design

27. EXECUTION PHASE • Availability of Clinical and PK Data • Availability of the clinical data is important in order to ensure that DMC has sufficient up to date data to make a correct recommendation • Sites need to be informed that this is an adaptive study and hence the data need to be entered in the electronic data capture as soon as it is available and if PK data plays a role in DMC recommendation, then these data need to be available as soon as they are collected • It is advised that a mechanism is put in place to monitor the availability of data versus the expected data, so that the sites are informed when there are substantial discrepancies

28. EXECUTION PHASE • Cleaning of Data • The key is to identify the parameters that need to be cleaned as soon as they are available, for example the end points that are used in the adaptive design software, entry of randomization numbers into the database, safety parameters of interests and any other key data that impacts DMC recommendation • Updating Randomization Scheme • The adaptive design software should update the randomization system automatically - hence a utility is built for this purpose • The utility should also allow DMC recommendation to be implemented such as dropping an arm for safety that is not included in the allocation algorithm

29. EXECUTION PHASE • Impact on Drug supply • With an adaptive design, the drug supply requirements at a given site are less predictable. Thus, it is a particular challenge to insure that the required treatment is available at the point of randomization • We need to avoid having to use an alternative treatment because the assigned treatment is not available (“forced randomization”) • Reproducibility • Each time that the interim reports are produced, the programs, inputs, logs, output, analysis datasets, seed number that was used to generate the adaptive randomization number and raw data need to be stored in the secure location • Everything required to be able to re-run the adaptive design and obtain the same outputs must be saved in order to ensure the reproducibility of the results in future

30. EXECUTION PHASE • Decision Outcome • It is understood that the DMC committee will serve strictly in an advisory capacity and its recommendations are non-binding • Formal implementation and communication of DMC recommendations will be managed by the sponsor steering committee (SSC) • It is SSC responsibility to endorse DMC recommendation and communicate their decision to the appropriate group

31. CLOSE OUT ACTIVITIES • Once the trial is completed all documents must be stored in a secure location so that they can be accessible for any audits

32. CONCLUSION • Learn phase clinical trials incorporate adaptive randomization have a number of advantages: • Potential to accelerate the identification of promising compounds • Improve the precision of the key dose-response information required for phase III success • Allow the efficient elimination of ineffective or poorly-tolerated compounds from additional testing • Conducting such trials introduces additional logistical complexity • Requires unblinded trial data be available in a timely fashion while maintaining data security • Interfacing of multiple software systems • Processes to address the information flow between DMC and other parties • In many situations the advantages of these designs justify the effort to deals with these logistical challenges