1 / 30

Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics

FHCRC 2014 Risk Prediction Symposium June 11, 2014. Illustration of the evaluation of risk prediction models in randomized trials Examples from women’s health studies. Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics Department of Obstetrics & Gynecology

ciaran-caldwell
Download Presentation

Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FHCRC 2014 Risk Prediction Symposium June 11, 2014 Illustration of the evaluation of risk prediction models in randomized trialsExamples from women’s health studies Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics Department of Obstetrics & Gynecology Academic Medical Center, University of Amsterdam, the Netherlands

  2. Clinical Problem IPre-eclampsia

  3. fullPIERS model Lancet, 2011

  4. Development Method • Patients: • 2000 women admitted in hospital for pre-eclapmsia (260 event) • Outcome: • Maternal mortality or other serious complications of pre-eclampsia • Logistic regression model with stepwise backward elimination

  5. Final model Logit P(D) = 2.68 – (0.054 × gestational age at eligibility) + (1.23 × chest pain or dyspnoea) – (0.027 × creatinine) + (0.21 × platelets) + (0.00004 × platelets2) + (0.01 × AST) – (0.000003 × AST2) + (0.00025 × creatinine × platelet) – (0.00007 × platelets × AST) – (0.0026 × platelets × SpO2)

  6. Performance of full-PIERS model Reported good risk discrimination and calibration

  7. Online calculator

  8. HYPITAT trial (2005-2008) • P Women at 36-41 wks of pregnancy with mild pre-eclampsia (n=750) • I Early Induction of labor (LI) • C Expectant monitoring (EM) • O Composite measure of adverse maternal outcomes

  9. HYPTAT Results (relative risk 0.71, 95% CI 0.59–0.86, p<0·0001)

  10. Modeling LogitP(D=1|T,Y) = β0+ β1T + β2Y + β3TY • D = 1 Adverse maternal outcome • Y = fullPIERS score • T = Treatment • 1 Labor induction • 0 Expectant monitoring

  11. FullPIERS for guiding labor induction fullPIERS score P for interaction: 0.93

  12. Clinical Problem IIPreterm birth

  13. Cervical pessary Medical device inserted to vagina to provide structural support to cervix

  14. ProTWIN trial (2009-2012) • P Women with multiple pregnancy (twin or triplet) between 12 & 20 weeks pregnancy • I Cervical Pessary (n = 403) • C Control (n = 410) • O Primary: Composite Adverse perinatal outcome

  15. ProTWIN Results (relative risk 0.98, 95% CI 0.69–1.39)

  16. Pre-specified subgroup analysis Cervical length (<38 mm vs >= 38 mm)

  17. Pre-specified subgroup analysis (P for interaction 0.01) Trial Conclusion: Clinicians should consider a cervical pessary in women with a multiple pregnancy and a short cervical length.

  18. Other Markers • Obstetric history (parity) • Nulliparous • Parous with no previous preterm birth • Parous with at least one previous preterm birth • Chorionicity • Monochorionic • Dichorionic • Number of fetuses • Twin • Triplet

  19. One marker at a time analysis

  20. Modeling LogitP(D=1|T,Y) = β0+ β1T + ΣβiYi+ ΣβjTYj • D = 1 composite poor perinatal outcome • Y = Markers • T= Treatment • 1 pessary • 0 control - Internal validation by bootstrapping

  21. Multi-marker model * Shrunken with an average shrinkage factor of 0.76 c-stat : 0,71 (95%CI: 0,66-0,77); optimism-corrected c-stat: 0,69 (95%CI: 0,63-0,74)

  22. How can the model be used in practice? • Calculating Risk without pessary • Using the model and setting treatment = 0 (control) • Calculating Risk with pessary • Using the model and setting treatment = 1 (pessary) • Calculating the predicted absolute benefit from pessary • Risk without pessary – Risk with pessary • Positive: woman benefits • Negative: woman does not benefit

  23. Predicted benefit from pessary

  24. Calibration of the predicted benefit

  25. Model performance • Multimarker positivity rate: • 35% (31-39%) • Benefit from pessary in multimarker-positives • 15% (7- 23%) • Benefit from no pessary in multimarker-negatives • 8% (3-13%) • Risk reduction by multimarker-based strategy • 10% (6-15%)

  26. Conclusion • Common assumption for application of risk prediction models for treatment selection: “Being at higher risk of outcome implies a larger benefit from treatment” • Not necessarily true • Developing models using trial data and modeling the interaction between markers and treatment might be a more optimal strategy

  27. Open Research Questions • Optimal modeling strategy? • Optimal algorithm for variable selection? • Optimal method for optimism correction?

  28. p.tajik@amc.nl Thanks! Any Questions?

  29. Multimarker vs. CxL only

  30. Two examples

More Related