Quantification and evaluation of risk at the time of licensing

1. Quantification and evaluation of risk at the time of licensing Dr David Wright Senior Statistical Assessor and Scientific Advice Coordinator MHRA

2. Contents • General observations • Methodological issues and complications • Example - CV risk in Diabetes • CHMP guideline • CHMP Scientific advice • Divergence between EU and FDA positions • Summary

3. General Considerations of evidence of safety in MAAs • Requirements less clear than for efficacy, at least from statistical (planning) perspective • Where does the burden of proof lie? • Sponsor to exclude / quantify risk? • Regulator to identify, and act on, signal? • What uncertainty is acceptable at time of licensing? How to weight uncertainty in the safety evaluation / benefit-risk decision? How to plan?

4. General Considerations of evidence of safety in MAAs • Studies powered for efficacy – study duration may consider safety, but the sample size rarely does • Composite endpoints for efficacy – rarely for safety • Inference too often based on data alone rather than data + pharmacology • Over-reliance on p-values to indicate presence / absence (sic) of effect, rather than point estimates and confidence intervals

5. General Considerations of evidence of safety in MAAs • Statistical methods apply equally to efficacy and safety data, even if inference / standards for assessment differ • Aim: to increase power without bias • Possible Methods: • More patients, longer trials! • Bayesian approach to account for clinical pharmacology • Composite endpoints • Meta-analyses • All of the above in addition to current standards

6. Methodological issues and complications • Power / Imprecise estimates • E.g. patient population in diabetes trials are at low CV risk → low event rate → low power. Thus sample size to detect / exclude (important) risk exceeds that required to show efficacy on glycaemic control. • Development programmes adequately powered for all AEs of interest would be excessively burdensome. • Planning – What objectives? What precision / delta? What sample size?

7. Methodological issues and complications • CPMP/EWP/2330/99 Points to Consider for Application with 1. Meta-Analyses, 2. One Pivotal Trial • An “accepted regulatory purpose for meta-analysis” is “to evaluate safety in a subgroup of patients, or a rare adverse event in all patients” • Pre-specification • Homogeneity of study design and study results • Consistent collection and evaluation of data • Discuss potential biases • Caution over dilution of effect

8. Methodological issues and complications • Sources of heterogeneity / bias • Patient populations (e.g. phase II vs phase III, active control vs placebo control data) • Trial durations • Different control arms (placebo, different actives) with potentially different underlying risk • Internal consistency across subgroups • Type I error considerations may differ for different claims i.e. ‘exclude important increase risk’ vs. ‘beneficial effect’

9. Example - CV risk in DiabetesCHMP guideline • CPMP/EWP/1080/00 Rev. 1 Guideline on clinical investigation of medicinal products in the treatment of diabetes mellitus • Some monotherapy data, not all ‘add-on’ • Little particular emphasis on CV risk over other risks. Doesn’t try to provide a tick list. • Representative population. Include those at high CV risk. • Use pooled analysis and / or specific study to detect “less common adverse events” • No particular delta to target

10. Example - CV risk in DiabetesCHMP guideline • “At the time of the MAA, the overall results of this safety programme should be submitted and discussed in terms of internal and external validity and clinical justification of the safety outcome. Acceptability of the data presented will be decided based on its overall quality, the point and interval estimates obtained for the calculation of specific risks, including cardiovascular risk compared to controls, and the reliability of these estimations. A summary of what is known about CV risk should be proposed for the SmPC.” • “Indications of increased risk of certain adverse events or unacceptable lack of precision are an important concern and may trigger the request for additional specific CV outcome trials to exclude an unacceptable increase in CV risk associated with the new agent before granting of MA.”

11. Example - CV risk in DiabetesScientific Advice • A number of procedures based on FDA Guidance for Industry Diabetes Mellitus – Evaluating CV risk in new anti-diabetic therapies to treat type 2 diabetes. • Pooled analysis appropriate, but consider heterogeneity of patient populations and control arms • FDA targets (1.8 and 1.3) represent agreeable planning assumptions, but the acceptability of this margin depends upon • the underlying CV risk associated each control group • the precise patient population recruited, • the estimated benefit and both the identified and potential risks of treatment, including any evidence for increased CV risk obtained from elsewhere in the development programme.

12. Example - CV risk in DiabetesScientific Advice • Pooling by dose not acceptable unless demonstrably conservative • Present absolute and relative risks • Submission based on interim analysis / Inclusion of an interim analysis of an ongoing study in a meta-analysis is acceptable, providing the trial is properly planned, conducted and analysed

13. Example - CV risk in DiabetesDivergence between EU and FDA positions • Similarities: • Same endpoint and need for independent endpoint committee • Meta-analysis and / or specific study • Include ‘high-risk’ population • Pre-specification • Internal consistency • Differences: • FDA guidance specific to CV risk – EU talks about ‘risk’ more generally • FDA guidance presents specific targets for delta. No formal EU position here, though a recognition that these are probably acceptable for planning purposes.

14. Summary • Interpreting safety data is complex, uncertainties will remain even with improved methodology • Increased use of statistical tools / approaches for safety data, including meta-analyses, at least in situations where otherwise estimates are imprecise and where programme design permits. • Important methodological considerations for design, analysis and interpretation • Should there be general (ICH?) guidance in the area of analysing and presenting safety data?