1 / 27

What EHRs Can Deliver that Randomized Clinical Trials Cannot

What EHRs Can Deliver that Randomized Clinical Trials Cannot. Retrospective studies with long-term follow-up Robert W. Grundmeier, MD July 13, 2009. Disclosures. No conflicts of interest No off-label uses of commercial products will be discussed. Overview.

quintessa
Download Presentation

What EHRs Can Deliver that Randomized Clinical Trials Cannot

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. What EHRs Can Deliver that Randomized Clinical Trials Cannot Retrospective studies with long-term follow-up Robert W. Grundmeier, MD July 13, 2009

  2. Disclosures • No conflicts of interest • No off-label uses of commercial products will be discussed

  3. Overview • Re-use of existing clinical data in electronic health records for research • The potential and challenges • Past experience with successful retrospective cohort studies in the EHR • Urinary tract infections • Asthma • The path ahead

  4. Before: Paper-Based “Poetry” Fabricated Chart...Based on a true story

  5. After: Detailed “Semi-Structured” EHR Template

  6. The Potential:Rich and Fresh Data • Thousands of repeated observations recorded for each potential subject over time • Longitudinal health problem diagnoses • Billing diagnoses • Vital signs and measurements • Prescriptions • Immunizations • Structured preventive health visits • Laboratory and radiology data • Procedures • And many more!

  7. The Potential:Large Volume of Data • 7 Years of data • 9 Subspecialty centers • 9 Subspecialty divisions • 32 Primary care sites • 300,000+ Patients • 2,500,000+ Visits • 65,000,000+ Observations

  8. Case Study #1 • Antibiotics for UTI prophylaxis (Conway PH. JAMA 2007) • Pediatric Asthma Hospitalizations and the Quality of Ambulatory Care

  9. Urinary Tract Infection (UTI) Study Cohort • Almost 75,000 subjects • 30 practices • 5 years of EHR data • Urine culture results from 3 laboratories • Hospital and specialty center radiology data • One Robert Wood Johnson Fellow

  10. Urinary Tract Infection (UTI) Study Findings • 12% annual incidence of recurrent UTI in children with an initial UTI • Significantly higher rates in children with high grades of vesico-ureteral reflux • Antimicrobial prophylaxis… • Did not change the rate of recurrent UTI • Increased prevalence of resistant organisms in recurrent UTI from 53% to 90%!

  11. Urinary Tract Infection (UTI) Study Challenges • Diagnosis codes only had “moderate” agreement with culture results • Kappa = 0.46 • Interpreting urine culture data required some natural language processing • Validation proved this approach superior to using diagnosis These kids really had a UTI

  12. Urinary Tract Infection (UTI) Study Challenges • Uncertainty about whether or not we had accurately identified the first UTI • Considered using a “birth cohort” with complete data in EHR • Instead chose to review all paper charts for patients with UTI (N=775) • 91 cases excluded due to documentation of prior UTI before EHR implementation • Only 1 case considered a “false positive”

  13. Case Study #2 • Antibiotics for UTI prophylaxis (Conway PH. JAMA 2007) • Pediatric Asthma Hospitalizations and the Quality of Ambulatory Care

  14. Asthma Study Questions: • Does quality of asthma care affect hospitalization rate? • Are there disparities in asthma healthcare? Methods: • Almost 6,000 subjects from 5 practices • 5 years of EHR data • 24 independent variables • 1 outcome (hospitalization) • One AHRQ Contract

  15. Asthma Study Preliminary Results

  16. Univariate Predictors of Asthma Hospitalization • Age < 4 years • .128 vs .063 hospitalizations per subject per month • Moderate to severe persistent asthma • .075 vs .044 hospitalizations per subject per month • African American Race • .072 vs .055 hospitalizations per subject per month • Public Insurance • .073 vs .065 hospitalizations per subject per month

  17. Asthma Study Challenges: Unmeasured Attributes • Marginally adequate socioeconomic status (SES) markers for retrospective studies • Public vs. private insurance is about as good as it gets • Geocoding may help • Median census tract income • Housing type

  18. Bad Luck: Simultaneous QI Efforts Inseparable

  19. Asthma “Misclassification across time and space” • Common conditions are coded commonly, and reasonably well • 57,820 Patients billed for asthma care • 53,824 Patients with asthma on problem list • 54,993 Patients with at least 2 albuterol prescriptions • This is EXCELLENT correlation

  20. Persistent Asthma • What about persistent asthma? • 16,949 Patients billed for “persistent asthma” • 11,943 With “persistent asthma” as a problem • But… • 23,673 Patients with at least 2 inhaled corticosteroid prescriptions which implies persistent asthma • And… • Only 3,553 With persistent symptoms based on questionnaire • Huh?

  21. Non-Random Misclassification By Care Location • “It is OK to compare organizations using their electronic data because everyone has the same problems with their data… the playing field is level” • Anonymous (Hospital Executive) • “Oh, really?” –Svetlana (CBMi Data Analyst)

  22. Non-Random Misclassification By Care Location (Mistake in Query) Svetlana: “…And do you really think that all the players will write their queries correctly?” WRONG!

  23. Non-Random Misclassification Over Time • And, the playing field changes over time • In 2004 one could have been lulled into a false sense of security over the reliability of encounter or problem list data… Actually, WE WERE!

  24. Good News! Statistical Magic for Missing Data • Asthma severity is correlated with many variables available in the EHR • Frequency, type, and dose of preventive treatment • Frequency of quick relief prescriptions • Frequency of oral steroid prescriptions • Hospitalizations • We recently imputed severity for the 20% of our population that is unclassified • The results were unbelievably accurate… like magic

  25. The Bottom Line • Retrospective studies can and should be done with EHR data captured for routine care • When data are suspicious or missing, look for corroborating evidence • You don’t know what you don’t know, until you read the charts • Find cohorts enriched in the disease, brew some strong coffee, and read! • “Pound the pavement,” go to where data are collected

  26. The Way Forward: Improve Data Collection This is Free! Work! Bonus! • Must think about how to make the clinician want to use the new data capture tool • We are doing a comprehensive decision support intervention regarding ear infections for this reason

  27. Thank You

More Related