70 likes | 246 Views
Classification of Emergency Department CT Imaging Reports using Natural Language Processing and Machine Learning. Efsun Sarioglu , Kabir Yadav, Meaghan Smith, Hyeong -Ah Choi. This project supported by the NIH National Center for Research Resources ( UL1RR031988 and KL2RR031987).
E N D
Classification of Emergency Department CT Imaging Reports using Natural Language Processing and Machine Learning EfsunSarioglu, Kabir Yadav, Meaghan Smith, Hyeong-Ah Choi This project supported by the NIH National Center for Research Resources (UL1RR031988 and KL2RR031987)
Background, Objective & Methods • Use of electronic medical record data for clinical research and quality improvement requires free-text data interpretation for outcomes of interest. • Natural language processing has shown promise for this purpose • To demonstrate real-world performance of a hybrid NLP-machine learning system for automated classification of radiology reports
Approach Overview • Multicenter review of consecutive CT reports obtained for facial trauma using a trained reference standard • Medical Language Extraction and Encoding (MedLEE) • WEKA 3.7.5 • Salford Systems CART 6.6
Results • Total reports: 3710 • Positive cases: 460 (12.4%) • Manual coding had high reliability • Kappa=0.97 [95% CI 0.94-0.99]
CART Decision Trees (50:50) NLP (9-node) Raw Text (8-node)
Classification Performance • Unexpectedly high performance of machine learning without NLP • Comparable to inter-rater performance and prior studies of inter-physician agreement • Comparable to prior real-world and simulation studies
Concluding Remarks • How’s it novel? • One of only a handful of real-world NLP studies using validated reference standard • Translating existing NLP and machine learning technologies to support CER • Next step: validation • Test approach using other clinical cases • Evaluate different features or classification algorithms