1 / 14

OHDSI updates

OHDSI updates. Martijn Schuemie. Method Evaluation. Method Evaluation. Time line OHDSI Method Evaluation Task Force Two posters at OHDSI Symposium 2017 About to submit paper to Harvard Data Science Review. Evaluation. Metrics Area under the ROC Coverage of the 95% confidence interval

lbenally
Download Presentation

OHDSI updates

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OHDSI updates Martijn Schuemie

  2. Method Evaluation

  3. Method Evaluation Time line OHDSI Method Evaluation Task Force Two posters at OHDSI Symposium 2017 About to submit paper to Harvard Data Science Review

  4. Evaluation • Metrics • Area under the ROC • Coverage of the 95% confidence interval • Mean precision • Mean Squared Error • Type I and II errors • Fraction non-estimable • Databases • IBM MarketScan CCAE • IBM MarketScan MDCR • JMDC • Optum PanTher Gold standard 200 real negative controls 600 synthetic positive controls Methods CohortMethod SelfControlledCohort CaseControl CaseCrossover SelfControlledCaseSeries

  5. Results http://data.ohdsi.org/MethodEvalViewer/ Key findings: All have poor coverage / type I error (without calibration) IPTW almost as bad as no adjustment PanTher biased for most methods due to observation time issues Best methods (highest AUC + highest precision after calibration) are self-controlled methods (specifically: MSCCS)

  6. Methods Library

  7. Methods Library • A core set of OHDSI R packages • For population-level estimation and patient-level prediction • Moving deployment to CRAN • Production-level • Unit tested • Evaluated in the Methods Evaluation • Well-documented https://ohdsi.github.io/MethodsLibrary/

  8. Book of OHDSI

  9. Book of OHDSI • Central knowledge repository • OHDSI community • CDM + Vocabulary + conventions (THEMIS) • ATLAS + R packages • Characterization + Population-level estimation + Patient-level prediction • Network studies • Can be used in classroom • Community developed through GitHub + bookdown • Could print in paper later https://github.com/OHDSI/TheBookOfOhdsi

  10. Network studies How to share study code

  11. Sharing study code What is shared? R package per study (with dependencies) How is it shared? Previously: StudyProtocolSandbox + StudyProtocols Now: OhdsiStudies using submodules https://github.com/OHDSI/OhdsiStudies

  12. Other ways to share code? Chan: how is this envisioned in FEEDER?

  13. Topics for next meeting ?

  14. Next workgroup meeting http://www.ohdsi.org/web/wiki/doku.php?id=projects:workgroups:est-methods Eastern hemisphere: January 9 3pm Hong Kong / Taiwan 4pm South Korea 5:30pm Adelaide 8am Central European time 7am UK time Western hemisphere: December 20 6pm Central European time 12pm New York 9am Los Angeles / Stanford

More Related