IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID

1. IMPROVING PATIENT’S ELECTRONIC HEALTH RECORD COMPREHENSION WITH NOTEAID Balaji Polepalli Ramesh, Thomas Houston, Cynthia Brandt, Hua Fang and Hong Yu

2. Outline • Background • The NoteAid System & Evaluation • Results • Discussion • Conclusion & Future Work

3. Background • Patients reading their clinical notes has the potential to • Enhance medical understanding • Improve healthcare management and outcome

4. Background “A patient with hx of active tobacco abuse, bronchitis, and psoriasis presented to ED earlier today with c/o SOB, mild wheezing, chest congestion and chills”

5. The NoteAid System • NoteAid system - automatically • Identifies clinically relevant concepts • Links concepts to their definitions

6. The NoteAid System Knowledge Resources (Medline Plus, UMLS, Wiki) NLP Approaches NoteAid The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.

7. The NoteAid System Knowledge Resources (Medline Plus, UMLS, Wiki) EGD – Acronym for Esophagogastroduodenoscopy. It is a test to examine the lining of the esophagus (the tube that connects your throat to your stomach), stomach, and first part of the small intestine. It is done with a small camera (flexible endoscope) that is inserted down the throat. EGD may be done if you have symptoms that are new, cannot be explained, or are not responding to treatment, such as: Black or tarry stool, Feeling full sooner than normal or after eating less than usual, Swallowing problems or pain with swallowing, heart burns and others. NLP Approaches NoteAid The patient will be scheduled for a repeat EGD in one year for surveillance purposes of Barrett esophagus. From a GI standpoint, we recommend to proceed with bariatric surgery. However, he will need to continue daily PPI administration to maximize acid reduction. Otherwise, there are no additional recommendations.

8. The NoteAid System • A knowledge resource • MedlinePlus • UMLS – Unified Medical Language System • Wikipedia • NLP approaches

9. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concept • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

10. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concepts • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

11. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concept • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

12. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concept • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

13. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concept • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

14. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concept • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

15. NLP Approach • Two system components – • Concept Identifier • Process input text and identify clinically relevant concept • Definition Locator • Fetch definitions from MedlinePlus, UMLS and Wikipedia COMPONENT 2 COMPONENT 1 Sentence Splitter Concept Mapper Concept Filter Definition Fetcher Input Mapped Split Mapped Knowledge Linked Text Phrases Sentences Text CONCEPT IDENTIFIER DEFINITION LOCATOR

16. Evaluation • Four NoteAid implementations • MedlinePlus • UMLS • Wikipedia • Hybrid (MedlinePlus+UMLS+Wikipedia)

17. Evaluation Data • From the Pittsburgh NLP repository • 20 Progress Note reports • 20 Discharge Summary reports

18. Subjects • The Amazon Mechanic Turk • Has shown to be reliable for medical annotations and evaluations • Recruited 64 subjects • 8 systems (4 implementations X 2 types of EHR notes) • 8 subjects per system • 59 subjects completed the evaluation

19. Demographic Information • Gender • Female – 23 (39%) • Male – 36 (61%) • Race • Asian – 34 (57.6%) • White – 23 (39%) • African American – 1 (1.7%) • Alaskan Native – 1 (1.7%) • Education • High School – 9 (15.3%) • Associates – 12 (20.3%) • Bachelors – 23 (39%) • Masters – 15 (25.4%)

20. Evaluation Process • Each subject read 20 EHR notes before and after NoteAid implementation • Randomly assigned to an implementation • Self-reported comprehension on a scale of 1 to 5

21. Readability • Flesch-Kincaid grade level (Grade Level)

22. Evaluation Data Statistics

23. Readability and Self-Reported Comprehension • Grade Level correlated with self-reported comprehension score (ρ=-0.47, p<0.05, Spearman rank correlation)

24. Results • Average comprehension scores before and after each implementation *p<0.05, Non-parametric Wilcoxon signed-rank test

25. Number of Concepts Identified • Total number of concepts that were recognized by three different NoteAid implementations

26. Readability • UMLS Definition – Coagulopathy “Hemorrhagic and thrombotic disorders that occur as a consequence of abnormalities in blood coagulation due to a variety of factors such as COAGULATION PROTEIN DISORDERS; BLOOD PLATELET DISORDERS; BLOOD PROTEIN DISORDERS or nutritional conditions” • Wiki Definition – Coagulopathy “Coagulopathy is a condition in which the blood’s ability to clot is impaired. This condition can cause prolonged or excessive bleeding, which may occur spontaneously or following an injury or medical and dental procedures. The normal clotting process depends on the interplay of various proteins in the blood”

27. Discussion • Text readability correlated with the comprehension scores • Wiki performed the best • Content coverage • Readability

28. Limitations • Limitation • Lay people not patients performed evaluation • Order-effect bias • Score subjects’ EHR note comprehension but not to what extent they accurately comprehended the EHR note content

29. Conclusion and Future Work • NoteAid improved EHR note self-reported comprehension • Future Work • Improve concept coverage and filtering • Evaluate quality of the definitions • Evaluate content comprehension • Evaluate system in which patient read their own EHR notes

30. Acknowledgements • 1R01GM095476 • University of Massachusetts Medical School • National Center for Advancing Translational Sciences of the National Institutes of Health under award number UL1TR000161

31. http://clinicalnotesaid.orgThank You and Questions