Symbolic and Machine Learning Methods for Patient Discharge Summaries Encoding

1. Symbolic and Machine Learning Methods for Patient Discharge Summaries Encoding Julia Medori CENTAL (Centre for Natural Language Processing) Université catholique de Louvain (Belgium) Séminaire du Cental - 17/12/2010

2. Overview • Problem outline • System structure • Extraction • Encoding • Extraction module • Encoding module • Machine learning methods • Experiments for features selection • Results • Symbolic methods description • Method 1: Morphological Analysis (MA) • Method 2: Extended lexical patterns (ELP) • Methods combination • Results • Conclusions

3. Introduction • Aim Build a (semi-)automated system for ICD-9-CM encoding • Collaboration CENTAL/Saint-Luc • Université catholique de Louvain (Belgium) • CENTAL : Centre for Natural Language Processing • Saint-Luc hospital : • team of 10 coders processes medical records : Extraction of medical acts and diagnoses  ICD-9-CMcodes • 85,000 patient’s stays encoded each year.

4. Manual encoding

5. Data • International Classification of Diseases -9th Revision-Clinical Modification (ICD-9-CM) • Hierarchy : • first 3 digits -> general category : 1,135 categories • Digits 4 and 5 -> specific diagnosis : 15,688 codes • Example :

6. Objectives • Design a coding help: • a tool that will suggest the most likely codes to be assigned to a patient’s medical record. • Why not a fully automated system? • Main source of information : Patient discharge summary (PDS) • PDS : letter, addressed to patient’s GP with no standard structure • 15-20% of the codes inferred from other sources from patient’s medical record (often scanned documents).

7. System structure Machine learning module Extraction Coding Code modification according to context and stats Context analysis + tagging PDS PDS + ordered list of codes Dictionaries and linguistic structures Preprocessing Manual checking Morphological processing ICD9CM + Inclusions Matching lists

8. Structure outline • 2 steps : • Extraction • Develop an extraction system able to extract information necessary to the encoding task : • Diagnoses, procedures, locations, dates, allergies, aggravating factors, etc. => Reading help tool. • Encoding • Extracted information => codes through a combination of statistical and symbolic methods.

9. Extraction • Develop specialized linguistic resources • Specialized dictionaries • Diagnoses and procedures <= ICD-9-CM + UMLS • Medications • Anatomy • Linguistic structure description • Diagnoses context (present, absent, probable, etc.) • Allergies and smoking • Dates • Weight and height

10. Example of linguistic structure graph Fracture de l’épaule => <MALINDET> Fracture de l’<ANAT>épaule</ANAT></MALINDET>

11. Extraction result

12. Structure outline • 2 steps: • Extraction • Develop an extraction system able to extract information necessary to the encoding task : • Diagnoses, procedures, localisations, dates, allergies, aggravating factors, etc. => Reading help tool. • Encoding • Extracted information => codes through a combination of statistical and symbolic methods.

13. Machine Learning • Encoding = categorization problem • Features = extracted phrases? • Classes = codes • Baseline method : Naive Bayes • Tool: Weka • Corpus : • 13,635 PDS from Digestive Surgery • 90% training set / 10% test set (1364 PDS) • Average number of codes per PDS: 6.2 • Trained 1 classifier per code occurring>5 times in the corpus : • 775 codes -> 775 classifiers • Limitation: 5% rare codes • attributes: kept only those co-occurring at least twice with the codes. • Measures: Precision and recall according to the probability returned by the Naive Bayes test.

14. Experiments • A series of experiments were conducted where attributes were variants of the extracted diagnoses and procedures after stemming. • Variants implied: • Kept original word order or not. • Ex: excisional biopsy bile duct • Or bile biopsy duct excisional • Included details like location, date, context. • Excisional biopsy • Each word of the extracted phrases is a feature • Excisional • Biopsy • Bile • Duct • Words and morphemes (together) composing the extracted phrase • Bile biopsy excision excisional duct • Words and morphemes (separately) composing the extracted phrase • Excisional biopsy bile duct • Excision biopsy bile duct • Values were 0 or 1 whether the attribute was in the text or not. • Values were the frequency of the attribute in the text.

15. Results • 3 best results when thresholding the list of results where the probability • returned by Naive Bayes = 1

16. Discussion • Limitations of the machine learning method: • 5% rare codes – not enough data to build a classifier for these codes • Need for annotated data means that these methods are unable to face changes in classifications • In these cases, we need to use symbolic methods Kevers Laurent et Medori Julia, Symbolic classification methods for patient discharge summaries encoding into ICD, In: Advances in Natural Language Processing, 7th International Conference on NLP, IceTAL 2010, Reykjavik, August 16-18, 2010, Lecture Notes in Artificial Intelligence, 2010, p. 197-208

17. Objective • Automatic encoding of PDS according to categories (first 3 digits) • Use of symbolic methods • No need for annotated data • Can assign rare codes (27% used 5 times or less) • Principle : • Make use of the nomenclature • Enrich it with other resources in French from UMLS (Unified Medical Language System)

18. Corpus • 19,692 patient discharge summaries (PDS) in French • General Internal Medicine • 150,116 codes (137,336 categories) • 6,029 distinct codes (895 categories) • Average = 7.6 codes/document (7 categories)

19. Method 1 (MA) – General Principle • Based on the rich morphology of medical language • Ex. Bronchoscopy: Fibroscopiebronchique = bronchoscopie par fibre optique • 2 steps process : • Extract phrases or terms describing diagnoses or procedures to be encoded • Encoding : match these terms to the right code.

20. Method 1 (MA) – Encoding • Bags-of-words : Words – stop words + morphemes + meaning ICD-9-CM PDS Fibroscopie bronchique Bronchoscopie par fibre optique fibroscopie bronchique fibro- fibre -scopie bronch- bronche -ique bronchoscopie par fibre optique bronch- bronche -scopie Similarity score

21. Method 1 (MA) – Results

22. Method 2 (ELP) – Generalprinciple • Developed by L. Kevers as designed for the Stratego project on parliamentary documents. • Symbolic method with less manual work • Use existing « terminological » resources • ICD-9-CM + UMLS • Two steps process • Automatic transformation of existing terminological resources into an extraction resource (only once) • Use extraction resource on documents for terms extraction and classification (for each document)

23. Method 2 (ELP) – build extraction resource (1) • For each ICD-9-CM term (= a class), the automatic processing implies : • Gather synonyms (UMLS) « dengue » → « dengues », « dengue fever », « infection by the dengue virus » • Parse complex compound expressions « Infectious and parasitic diseases » → « Infectious disease » → « Parasitic disease » • Transform initial term into Extended Lexical Pattern (ELP) • Stopwords :→ « infection <TOKEN> dengue virus » • Stemming : → « infect <TOKEN> dengue virus » • Allow insertions : → « infect <I> <TOKEN> <I> dengue <I> virus » • Add negative contexts patterns • Build the main transducer for text annotation

24. Method 2 (ELP) – Transducer & output • Transducer for class '061' Zona [[053]] extremement douloureux [[729]] gastroscopie [[Z44]] acide [[E96]] anemie normochrome normocytaire [[285]] sequellaires apicales droite (tuberculose [[137]] intestin grele [[Z45]] tuberculose [[V12]] • Output of main transducer for a document oesophagite moderee aspecifique [[947]] infection a mycobacterie [[031]] fond de oeil [[Z16]] pas de [[-]] atteinte du nerf [[957]] zona [[053]] hyperthyroidie [[242]] goitre [[706]] goitre [[240]]

25. Method 2 (ELP) – Class assignment (2) • For a text to classify, analyse the main transducer output • When negative contexts, the phrase is skipped • Each recognized phrase has one (or more) related code • Compute a weight for each phrase based on • Frequency • Is a multi word expression (frequency*2), or not • Compute a weight for each code by summing up the weights obtained for the phrases • Result : ordered list of codes (possibly threshold it)

26. Method 2 (ELP) – Results

27. Combination of methods 1 & 2 • Merge the lists from method 1 & 2 • Threshold(M.1 union M.2) • Threshold(M.1 inter M.2) • Threshold(M.1) union Threshold(M.2) • Threshold(M.1) inter Threshold(M.2) • The weight for each method can be balanced • Example: 0.4*M.1 union 0.6* M.2

28. Evaluation of symbolic methods combination

29. Conclusions • Results have to be put into perspective: • Inter-annotator agreement ~70% • 15 to 20% cannot be inferred from PDS • Machine learning methods performed well. • Symbolic methods: • MA method based on extraction module : 66% of useful information is extracted. • ELP method performs better when built from short unambiguous phrases. ICD-9-CM code descriptions are more complex. • Future work : • Give more weight to information contained in important parts of the PDS (introduction, conclusion…) • Evaluate the actual help given to human coders • Combine with learning algorithms