1 / 28

Combining Information From Randomized and Observational Data: A Simulation Study

Combining Information From Randomized and Observational Data: A Simulation Study. Eloise E. Kaizar The Ohio State University. Joel Greenhouse Howard Seltman Carnegie Mellon University. June 5, 2008. TexPoint fonts used in EMF.

abla
Download Presentation

Combining Information From Randomized and Observational Data: A Simulation Study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Combining Information From Randomized and Observational Data: A Simulation Study Eloise E. Kaizar The Ohio State University Joel Greenhouse Howard Seltman Carnegie Mellon University June 5, 2008 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAA

  2. Outline • Motivating Example • Association between suicidality and antidepressant use in pediatric population • Trying to answer the right question • Exploiting strengths of different data • Simulation Study

  3. Pediatric Antidepressant Use • Problem: Antidepressant use may cause suicide for some children/adolescents • Goal: Estimate the average treatment effect for use in regulatory decision making

  4. Randomized Controlled Trials • Hammad, et al. (2006) Archives of General Psychiatry

  5. The Right Question • Study population average treatment effect • Population average treatment effect

  6. Heterogeneity • Variation due to differences in population (“True”) • Variation due to differences in study design (“Artifactual”)

  7. Evidence for Weak External Validity • Administrative data • Show no significant association between antidepressant use and suicidal actions (Valuck et al. 2004, Jick et al. 2004) • Epidemiological data • Suggest inverse relationship between antidepressant use and completed suicide • Geographically (Gibbons et al. 2006, Isacsson 2000, Ludwig and Marcotte 2005) • Temporally (Gibbons et al. 2007, Olfson, et al 1998)

  8. Assessing External Validity • Compare the RCT patients with a nationally representative probability sample of adolescents • Youth Risk Behavior Survey (YRBS) • Representative of adolescents attending school (aged 12-18) • Basic demographic information • Self-report depression • Self-report suicidality

  9. Match RCTs and YRBS • Consider only MDD RCTs of ages 12-18 • Consider only YRBS respondents reporting depression • Poststratify YRBS to match RCTs

  10. Compare Outcomes • 8-week suicidality • RCTs 3.6% • YRBS 7.1% • Suicide attempt • RCTs 5.4% (lifetime) • YRBS 19.9% (12-month)

  11. Randomized Controlled Trials • Hammad, et al. (2006) Archives of General Psychiatry

  12. Generalizing RCT Data • Reduce the size of the excluded population • Practical Clinical Trial • Estimate the effect size in the excluded population

  13. Current Approaches to Estimating Average Effect Size • Use meta-analysis to combine RCT data • Assume effect is not systematically heterogeneous by exclusion criteria • Use multi-level meta-analysis to combine RCT and observational data • Partial exchangeability • Assumes the mean is of interest • Include bias parameters

  14. Proposed Approaches • Confidence Profile Method [Eddy, et al., 1988, 1989] • Model the biases in observational and RCT data • Response Surface Approach [Rubin, 1990, 1991] • Create a response surface that incorporates design variables • Extrapolate to the ideal design • Cross Design Synthesis [US GAO, 1992] • Stratify data based on design variables • Extrapolate to empty cells

  15. Usefulness of Evidence RCT Ideal Stronger Internal Validity Obs. Weaker Stronger Weaker External Validity

  16. Framework

  17. Framework Weighted Integral

  18. Framework Randomized Experiments Observational Studies

  19. Linear Bias Model

  20. Simulation Study • Goal: • Use simulation study to investigate effectiveness of different methods for combining information from diverse sources in a realistic setting • Key characteristics: • 24 high-quality experiments with complete compliance and uniform randomization eligibility • 200 subjects, individual data unavailable • 1 high-quality observational study with no generalizability bias • 25,000 subjects, individual data available

  21. Simulation Study:Implementation • Generate 1000 data sets • Fit models using Bayesian approach • Compare on MSE, bias and coverage

  22. Linear Bias Model

  23. Summary • Even RCTs that have no heterogeneity may not be estimating the effect of interest. • Observational data may be used to assess the extent of the generalizability problem • The Cross Design Synthesis approach can potentially be effective for estimating average effect size • Still at the beginning of this work • More fair comparisons • Extend to real settings

More Related