330 likes | 641 Views
FHCRC 2014 Risk Prediction Symposium June 11, 2014. Illustration of the evaluation of risk prediction models in randomized trials Examples from women’s health studies. Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics Department of Obstetrics & Gynecology
E N D
FHCRC 2014 Risk Prediction Symposium June 11, 2014 Illustration of the evaluation of risk prediction models in randomized trialsExamples from women’s health studies Parvin Tajik, MD PhD candidate Department of Clinical Epidemiology & Biostatistics Department of Obstetrics & Gynecology Academic Medical Center, University of Amsterdam, the Netherlands
fullPIERS model Lancet, 2011
Development Method • Patients: • 2000 women admitted in hospital for pre-eclapmsia (260 event) • Outcome: • Maternal mortality or other serious complications of pre-eclampsia • Logistic regression model with stepwise backward elimination
Final model Logit P(D) = 2.68 – (0.054 × gestational age at eligibility) + (1.23 × chest pain or dyspnoea) – (0.027 × creatinine) + (0.21 × platelets) + (0.00004 × platelets2) + (0.01 × AST) – (0.000003 × AST2) + (0.00025 × creatinine × platelet) – (0.00007 × platelets × AST) – (0.0026 × platelets × SpO2)
Performance of full-PIERS model Reported good risk discrimination and calibration
HYPITAT trial (2005-2008) • P Women at 36-41 wks of pregnancy with mild pre-eclampsia (n=750) • I Early Induction of labor (LI) • C Expectant monitoring (EM) • O Composite measure of adverse maternal outcomes
HYPTAT Results (relative risk 0.71, 95% CI 0.59–0.86, p<0·0001)
Modeling LogitP(D=1|T,Y) = β0+ β1T + β2Y + β3TY • D = 1 Adverse maternal outcome • Y = fullPIERS score • T = Treatment • 1 Labor induction • 0 Expectant monitoring
FullPIERS for guiding labor induction fullPIERS score P for interaction: 0.93
Cervical pessary Medical device inserted to vagina to provide structural support to cervix
ProTWIN trial (2009-2012) • P Women with multiple pregnancy (twin or triplet) between 12 & 20 weeks pregnancy • I Cervical Pessary (n = 403) • C Control (n = 410) • O Primary: Composite Adverse perinatal outcome
ProTWIN Results (relative risk 0.98, 95% CI 0.69–1.39)
Pre-specified subgroup analysis Cervical length (<38 mm vs >= 38 mm)
Pre-specified subgroup analysis (P for interaction 0.01) Trial Conclusion: Clinicians should consider a cervical pessary in women with a multiple pregnancy and a short cervical length.
Other Markers • Obstetric history (parity) • Nulliparous • Parous with no previous preterm birth • Parous with at least one previous preterm birth • Chorionicity • Monochorionic • Dichorionic • Number of fetuses • Twin • Triplet
Modeling LogitP(D=1|T,Y) = β0+ β1T + ΣβiYi+ ΣβjTYj • D = 1 composite poor perinatal outcome • Y = Markers • T= Treatment • 1 pessary • 0 control - Internal validation by bootstrapping
Multi-marker model * Shrunken with an average shrinkage factor of 0.76 c-stat : 0,71 (95%CI: 0,66-0,77); optimism-corrected c-stat: 0,69 (95%CI: 0,63-0,74)
How can the model be used in practice? • Calculating Risk without pessary • Using the model and setting treatment = 0 (control) • Calculating Risk with pessary • Using the model and setting treatment = 1 (pessary) • Calculating the predicted absolute benefit from pessary • Risk without pessary – Risk with pessary • Positive: woman benefits • Negative: woman does not benefit
Model performance • Multimarker positivity rate: • 35% (31-39%) • Benefit from pessary in multimarker-positives • 15% (7- 23%) • Benefit from no pessary in multimarker-negatives • 8% (3-13%) • Risk reduction by multimarker-based strategy • 10% (6-15%)
Conclusion • Common assumption for application of risk prediction models for treatment selection: “Being at higher risk of outcome implies a larger benefit from treatment” • Not necessarily true • Developing models using trial data and modeling the interaction between markers and treatment might be a more optimal strategy
Open Research Questions • Optimal modeling strategy? • Optimal algorithm for variable selection? • Optimal method for optimism correction?
p.tajik@amc.nl Thanks! Any Questions?