Drug Safety Assessment and Data Mining

1. Drug Safety Assessment and Data Mining F. Gavini (IRIS, Servier), G. Le Teuff (Keyrus Biopharma) Journées Maths-Industrie ENSAI 4 mai 2010

2. Plan • Drug Safety Assessment • Knowledge Discovery in Database & Data Mining • Use of Data Mining in Drug Safety Assessment • An Example in Oncology

3. Drug Safety Assessment Drug safety assessment is an important goal in the drug development and Post Marketing Surveillance (PMS) It contributes to the balance of benefits and risks of the product It consists generally in anticipating, assessing and minimizing the cases of adverse drug reactions

4. Drug Safety Assessment At each step of the drug development Pre clinical Phase 1, 2-3 PMS (Post Marketing Surveillance) Mainly based on Spontaneous Reporting Systems Specific studies

5. Drug Safety AssessmentPre clinical studies Objective : to assess toxicity in animal Acute toxicity (LD50, …) Subchronic toxicity (13 / 26 weeks) Chronic toxicity and cancerogenesis Reproductive testing Statistical analysis Limits

6. Drug Safety AssessmentPhase 1 Objective Assess safety or toxicity (depending on compound) Define therapeutic window / Maximum Tolerated Dose Available data Few volunteers (or patients) / dose escalating process Thorough clinical assessment Statistical analysis Limits

7. Drug Safety AssessmentPhase 2-3 Objective : to assess safety of study drug in larger studies in patients vs. placebo or reference drug General safety parameters Adverse events (AE) / serious adverse events (SAE) , … Biology / Biochemistry / ECG / Vital signs, … Disease / class specific safety parameters e.g. CV safety in diabetes, ECG in QT prolonging drugs, … The AE are coded using dictionary MedDRA®

8. Drug Safety AssessmentPhase 2-3 Use of MedDRA® ( Medical Dictionary for Regulatory Activities)

9. Drug Safety AssessmentPhase 2-3 Statistical analysis (adverse events) ICH descriptive tables (counts, crude incidence, incidence rate, …) Number of AE (NAE) in a given primary system organ class or preferred term Number of patients (n) with at least one AE in a given preferred term or a given primary system organ class

10. Drug Safety AssessmentPhase 2-3 PRIMARY SOC /PREFERRED TERM Treat A(N=851) Treat B(N=846) NAE(1) n(2) %(3) NAE(1) n(2) %(3) ALL 82 76 9.0 74 62 7.3 Cardiac disorders 9 9 1.1 13 13 1.5 Myocardial infarction 4 4 0.5 6 6 0.7 Angina unstable 0 0 0.0 2 2 0.2 Cardiac failure 0 0 0.0 2 2 0.1 Cardiac failure acute … … … … … … …/… …/…

11. Drug Safety AssessmentPhase 2-3 • Statistical analysis (adverse events) • Inferential statistic • Adjusted odds ratios (Logistic regression) • Time to first event (Log-rank / Cox model) • Number Needed To Harm (NNTH) … • Limits • Repeated statistical testing or confidence intervals • Trial size, event incidence, trial population

12. Drug Safety Assessment Integrated Analysis Safety (IAS) Objective : to integrate all phase 2 and 3 trials Analysis of safety : estimate safety across clinical trials Adverse events / serious adverse events, … Biology / Biochemistry / ECG / Vital signs, … Statistical analysis ICH descriptive tables / confidence intervals Meta analysis vs naïve pooling Limits Updates

13. Drug Safety AssessmentPMS Key questions Which drugs (or combinations) induces which event ? Which patients are likely to experience the event (and which replacing therapy then) ? Definitions Adverse drug reaction (ADR) Spontaneous Report / Pharmacovigilance (PV) Detailed Case Reports PV Spontaneous Reporting Databases (SRD), PV Spontaneous Reporting Systems

14. Drug Safety AssessmentPMS Databases FDA Spontaneous Report System : PMS of all drugs since 1969 Data in public domain FDA Adverse Event Reporting System (AERS) Replaced SRS New AE coding system – MedDRA ® 97 Others : VAERS / Medical Devices databases WHO (World Health Organization) database Including drugs marketed outside US 67 countries National databases, …

15. Drug Safety AssessmentPMS Adverse events database limitations No protocol research No denominator Under-reporting in general / linked with drug and event Errors / Missing data / Duplicates Report rates change over time - Multiple drugs, multiple events, … Causal links ?

16. Drug Safety AssessmentRisk Management Plan (RMP) Objective Recently, Health Authorities suggest the laboratories to conduct a Risk Management Plan throughout the lifetime of a medicinal product Guideline on risk management systems for medicinal products for human use. EMEA/CHMP/96268/2005 This plan includes the pre-authorisation phase

17. Knowledge Discovery from Database and Data Mining With the growing of database, « classical » analysis of data become more and more difficult Problematics are more and more complex « The curse of dimensionality », [Bellman] Emergence of a new concept : KDD and Data Mining International Conferences on KDD and DM (since 1995) Data Mining and Knowledge Discovery Journal (1997)

18. Knowledge Discovery from Database and Data Mining KDD was initiated in the early 90’s [Piatteski-Shapiro] Concept «The notion of finding useful patterns (or nuggets of knowledge) in raw data has been given various names, including knowledge discovery in data bases, data mining, knowledge extraction, information discovery, information harvesting, data archaeology, and data pattern processing» [Fayyad et al., 1996] Objective In practise, making decisions and discovery new knowlegde

19. Knowledge Discovery from Database and Data Mining Definition The term « Data Mining » has been used by the statisticians, data analysts While the term « KDD » has been mainly used by the searchers in artificial intelligence and automatic learning Convergence from multiple domains Database, Data Analysis, Statistic, Neural Networks, Visualization, Automatic learning

20. Knowledge Discovery from Database and Data Mining Steps of KDD [Fayyad et al.1996] from data to knowledge Acquisition of data Creation of target data set Data cleaning and preprocessing Data reduction and projection Definition of tasks Choose of appropriate algorithms Data mining Mined patterns (interpretation) Test and validation of knowledge discovery

21. Knowledge Discovery from Database and Data Mining Data Mining versus statistical analysis • Data Mining • At the origin, type of approach: expert system • Several techniques • Decisional use • Few hypotheses on the data • Large database • Statistical analysis • Protocol and pre-specifications • Sampling representativity • Hypotheses testing • Model assumptions • Goodness of fit assessment • Not adapted to deep exploration of highly multi-dimensional database

22. Knowledge Discovery from Database and Data Mining What types of problems ? Classification Prediction Association With what method ? Statistic or not statistic With or without a priori hypotheses

23. Knowledge Discovery from Database and Data Mining Overview of the techniques Kmeans Neural network Association rules Decision tree

24. Use of Data Mining in Drug Safety Assessment: Introduction Data Mining is now recognized as a complementary approach by regulatory agenciesin pharmacovigilance Eudravigilance expert working group (EV-EWG). EMEA/106464/2006 rev.1. 2008 More recently, Health Authorities invite laboratories to use data mining and require a risk management plan (RMP) This plan should be conducted as a continuing process throughout the lifetime of a medicinal product, including the pre-authorisation phase

25. Data Mining & PMS • Signal detection / WHO ‘Reported information on a possible relationship between an adverse event and a drug, the relationship being unknown or incompletely documented previously’ • Role • Change in specific drug-event reporting pattern • Comparison of drug-event reporting vs other drugs (same class)

26. Frequentist methods Measure of Signal disproportionate reporting : Reporting Ratios and Proportional Reporting Ratios Measures of discrepancy : CHI-2 Kullback-Leibler Data Mining & PMS

27. Two-way drug-AE table Observed Expected True Drug 1 Drug 2 … Drug C All Drugs Event 1 n11 n12 n1c n1. Event 2 n21 n22 n2c n2. … … … nij … … Event r nr1 nr2 nrc nr. All events n.1 n.2 n.c n.. Data Mining & PMS = marginal probabilities

28. FREQUENTIST RR and PRR Estimates Data Mining & PMS Evans 2000

29. Data Mining & PMS FREQUENTIST • Definition of Signal / (unexpected large cell) • Discrepancy Vs.

30. BAYESIAN SHRINKAGE METHODS Bate and al. 1998 Bayesian confidence propagation by neural network (BCPNN) DuMouchel 1999 Empirical Bayes Gamma-Poisson Model (EBGM) DuMouchel and Preibon 2001 Multi-item Gamma-Poisson Shrinkage (MGPS) Data Mining & PMS

31. BCPNN METHOD Information component Model assumptions Parameters / prior distributions Data Mining & PMS

32. BCPNN METHOD Posterior distr. of all p follow a Beta distribution Etimate Variance (CI) based on posterior distribution Data Mining & PMS • Thresholds • - Lower 95%CI bound >0 • - Sudden increase of 1 over 3 months

33. EXAMPLE Bates & Evans 2009 Cerivastatin-rhabdomyolysis WHO T4 1998 IC: 1.90 - 95% : [0.44-3.36] WHO T1-T2 99 IC: 1.88 to 3.30 Data Mining & PMS

34. Data Mining & PMS Empirical Bayes Gamma-Poisson Model (EBGM) • Parameter of interest • Prior • Posterior • EBGM05 5% lower bound > 2 Signal • Stratification

35. Data Mining & Clinical Safety • Just at its starting point • Literature • Even if databases are not very large • Need in Clinical trials / Safety assessment • alternative to descriptive / repeated tests on hundreds of PT • Visual needs

36. Data Mining & Clinical Safety • Visual Data Mining - identify patterns / rules for patients with AE Algorithms / rules Overlearning assessment - Use of several dimensions of safety (AE, biology, biochemistry)

37. Data Mining & Clinical Safety Use medDRA hierarchy / hierarchical models Use of multiple decision trees Appropriate use of Neural networks  Southworth & O’Connell 2009 Use of bayesian networks e.g. oncology phase I

38. An Example in Oncology • Objective • Find the maximum tolerated dose (MTD), • and establish the recommended phase II dose of chemotherapy

39. An Example in Oncology • Terminated trials, +new protocols • 4 Dose-escalation Phase 1 trial of an IV administration of X in liquid / solid tumors • Prospective, non-randomised, non-comparative, open-label studies • A traditional Carter algorithm-based design : ‘3+3’ design as shown in the figure below • Dose Limiting Toxicity (Thrombocytopenia)

40. Dose Level i + 1 An Example in Oncology • Design Dose is the MTD Dose Level i  2/6  2/3 Total DLTs ? 1/3 Dose 3 more patients Dose 3 patients DLTs ? 0/3 =1/6 Dose is safe Dose is safe

41. An Example in Oncology • Population set • N= 105 patients from 4 Phase 1 studies (N1=18, N2=30, N3=15 and N4=42) • With advanced solid / liquid tumours • Dose ranging : from 20 to 80 mg/m²by 10 • Outcome (target variable): toxicity (0/1) • Covariates: dose, age, type of tumor (solid, liquid), nb of days off in cycle, platelets at baseline, race

42. An Example in Oncology • Covariates dose, age, type of tumor (solid, liquid)*, nb of days off in cycle (1 vs. 2)*, platelets at baseline, Race (3 races) *depends on to study

43. An Example in Oncology • Statistical analysis • Conventional analysis by trial • Bayesian network: use conditionnal probability Continuous covariates will be categorized • New protocols : help for decision making

44. An Example in Oncology • Bayesian networks • Graph theory and probability theory • Bayes theorem: • Use of discrete variables

45. An Example in Oncology • Oriented graph • Nodes : variables • Oriented graphs : condition dependances • Conditional independance: Chemotherapy is independant of ‘Smoking’ conditional to Cancer=‘Yes’ Smoking Lung Cancer Chemo

46. An Example in Oncology • Probabilities associated to nodes conditional to parents • Find all joint and conditional probabilities Lung Cancer Chemo Smoking P(Smoking) P(Cancer|Smoking) P(Chemo|Cancer)

47. An Example in Oncology • Bayesian networks • Bayesian network: use conditional probability Supervised Networks Learned from data • Naive approach • Markov approach Supervised by Expertise • Expert approach Network Propagation

48. An Example in Oncology (6/7) • Results

49. An Example in Oncology (7/7) • Results

50. An Example in Oncology (8/7) • Results