1 / 30

OVERVIEW

STATISTICAL CONSIDERATIONS IN POSTMARKETING SAFETY EVALUATION A. Lawrence Gould Merck Research Laboratories West Point, PA [goulda@merck.com] FDA/Industry Workshop 29 September 2006 Washington, DC. OVERVIEW. Spontaneous AE Reports.

amandla
Download Presentation

OVERVIEW

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. STATISTICAL CONSIDERATIONS IN POSTMARKETING SAFETY EVALUATIONA. Lawrence GouldMerck Research LaboratoriesWest Point, PA [goulda@merck.com]FDA/Industry Workshop29 September 2006Washington, DC

  2. OVERVIEW

  3. Spontaneous AE Reports • Clinical trial safety information is limited & relatively short duration • Safety data collection continues after drug approval • Detect rare adverse events • Obtain tolerability information in a broader population • Large amount of low-quality data collected • Not usable for trt comparisons or risk assessment • Unknown sensitivity & specificity • Evaluation by skilled clinicians & epidemiologists • Long history of research on issue

  4. Information Available Postmarketing • Previously undetected adverse and beneficial effects that may be uncommon or delayed, i.e., emerging only after extended treatment • Patterns of drug utilization • Effect of drug overdoses • Clinical experience with study drugs in their “natural” environment

  5. The Pharmacovigilance Process Traditional Methods Data Mining Detect Signals Generate Hypotheses Insight from Outliers Public Health Impact, Benefit/Risk Refute/Verify Type A (Mechanism-based) Estimate Incidence Act Inform Type B (Idiosyncratic) Restrict use/ withdraw Change Label

  6. Considerations & Issues (An Incomplete List!) • Incomplete reports of events, not reactions • Bias & noise in system • Difficult to estimate incidence because no. of pats at risk, pat-yrs of exposure seldom reliable • Significant under reporting (esp. OTC) • Synonyms for drugs & events → sensitivity loss • Duplicate reporting • No certainty that a drug caused the reaction reported • Cannot use accumulated reports to calculate incidence, estimate drug risk, or compare drugs

  7. DATA MINING

  8. Data Mining is a Part of Pharmacovigilance • Identify subtle associations (e.g., drug+drug+event) and complex relationships not apparent by simple summary • Identify potential toxicity early • Finding ‘real’ D-E associations similar to finding potential active compounds or expressed genes – not exactly the same (no H0) – more like model selection • Still need initial case review respond to reports involving severe, potential life-threatening events eg., Stevens-Johnson syndrome, agranulocytosis, anaphylactic shock • Clinical/biological/epidemiological verification of apparent associations is essential

  9. Typical Data Display Basic idea: Flag when R = a/E(a) is “large” • Some possibilities • Reporting Ratio: E(a) = nTD nTA/n • Proportional Reporting Ratio: E(a) = nTD c/nOD • Odds Ratio: E(a) = b  c/d • Need to accommodate uncertainty, especially if a is small • Bayesian approaches provide a way to do this

  10. Currently Used Bayesian Approaches • Empirical Bayes (DuMouchel, 1998) & WHO (Bate, 1998) • Both use ratio nij / Eij where nij = no. of reports mentioning both drug i & event j Eij = expected no. of reports of drug i & event j • Both report features of posterior dist’n of ‘information criterion’ ICij = log2 nij / Eij = PRRij • Eij usually computed assuming drug i & event j are mentioned independently • Ratio > 1 (IC > 0)  combination mentioned more often than expected if independent

  11. Comparative Example (DuMouchel, 1998) • No. Reports = 4,864,480, Mentioning drug = 85,304

  12. DATA MINING EXAMPLES INCORPORATING STATISTICAL REFINEMENTS

  13. Result From 6 Years of Reports on Lisinopril Events w/Lower 5% RR Bnd > 2 (Bold  N  100)

  14. Persistence (& Reliability) of Early Signals

  15. Accumulating Information over Time • Lower 5% quantiles of RR stabilized fairly soon

  16. Time-Sliced Evolution of Risk Ratios • See how values of criteria change over time within time intervals of fixed length Change in ICij for reports of selected events on A2A from 1995 to 2000 tension = hypotension failure = heart failure kalemia = hyperkalemia edema = angioedema

  17. Masking of AE-Drug Relationships (1) • Company databases smaller than regulatory databases, more loaded with ‘similar’ drugs eg, Drug A is 2nd generation version of Drug B, similar mechanism of action, many reports with B • Elevated reporting frequency on Drug B could mask effect of Drug A • May be useful to provide results when reports mentioning Drug B are omitted

  18. Masking of AE-Drug Relationships (2)

  19. Example 2: Vaccine-Vaccine Interaction • From FDA VAERS database, reports from 1990-2002 • Intussusception is a serious intestinal malady observed to affect infants vaccinated against rotavirus • Look at reports of intussusception that mention rotavirus vaccine (RV) and DTAP vaccine • DTAP is a benign combination vaccine commonly administered to infants • Demonstration question: Intussusception very commonly reported with RV – but does the reporting rate depend on whether DTAP was co-administered? • Not easy to address using standard pharmacovigilance procedures

  20. Outline of Analysis • Standard tools provide intussusception reporting rate for pairs of vaccines, and for vaccines singly • Result is a 3-way count table (corresponding to RV + or -, DTAP + or -, and intussusception + or -) • Use log-linear model to see if intussusception is mentioned with the two vaccines together more often than the separate vaccine-intussusception reporting associations would predict • Turns out that there is an association – Likelihood ratio chi-square is 17.41, 1 df, highly significant

  21. Observed and Expected Report Rates

  22. Comments • Intussusception seems to be reported more often than expected when RV and DTAP are given together than when RV is given without DTAP, after adjusting for individual vaccine-intussusception associations • Reports of intussception without RV are very rare, about 4.5/10,000 reports if RV is not mentioned • The joint effect of RV and DTAP on intussusception reporting is small, but does reach statistical significance • Not clear that apparent association means anything -- actual synergy between RV and DTAP seems unlikely, but explanation requires clinical knowledge

  23. A NEW BAYESIAN APPROACH(Gould, Biometrical Journal 2006, to appear)

  24. Model for Process Generating Observations • ni = no. of reports mentioning i-th drug-event pair ~ Poisson (true for EB approach as well) f(ni | Ei, i) = fPois(ni ; iEi) • i drawn from a gamma(a0, b0) distribution or from a gamma(a1, b1) distribution • A model selection problem • Dist’ns reflect physician/epidemiologist’s judgment as to what range of  values corresponds to ‘signals’, and what does not Expected count under independence Association measure

  25. Prior/Model Density of  • Bayes approach starts with a random mixture of gamma densities, f0( ; , a0, b0, a1, b1) = (1 - )fgam(; a0, b0) + fgam(; a1, b1) Use value of Ppost(g = 1) for inference • EB approach starts with expectation wrt  given p  nonrandom mixture of gamma densities, f0( ; p, a0, b0, a1, b1) = pfgam(; a0, b0) + (1-p)fgam(; a1, b1) Use quantiles of posterior dist’n of  for inference Analyst specifies parameter values Data determine parameter values

  26. Comments • Bayes and EB approaches both model strength of drug-event reporting assn as a gamma mixture • Diagnostic properties of Bayes method can be determined analytically or by simulation • Unknown separation of the true alternative dist’ns for  more important than prior dist’n used for analysis • Methods described here can be applied to other models – Scott & Berger (2005) used normal distributions – could also use binomial instead of Poisson, beta instead of gamma distributions to develop screening methods for AEs in clinical trials

  27. DISCUSSION

  28. Discussion • Bayesian approaches may be useful for detecting possible emerging signals, especially with few events • MCA (UK) currently uses PRR for monitoring emergence of drug-event associations • Signal detection combines numerical data screening, statistical interpretation, and clinical judgement • Most apparent associations represent known problems • ~ 25% may represent signals about previously unknown associations • The actual false positive rate is unknown

  29. What Next? • PhRMA/FDA working group has published a white paper addressing many of these issues Drug Safety (2005) 28: 981-1007 • Further refine methods, look for associations among combinations of drugs and events, timing of reports • Data mining is like screening, need to evaluate diagnostic properties of various approaches • Need good dictionaries: many synonyms  difficult signal detection • Event names: MedDRA may help • Drug names: Need a common dictionary of drug names to minimize dilution effect of synonyms

  30. Data Used to Construct Plot

More Related