1 / 32

Penny Hatcher, Supervisor and Grant Director Yaoli Li, CDC EHDI Coordinator

Quality Assessment of Minnesota Newborn Hearing Screening Data: A Pilot Study National EHDI Conference March 26, 2007 Salt Lake City. Penny Hatcher, Supervisor and Grant Director Yaoli Li, CDC EHDI Coordinator Nicole Brown, HRSA UNHSI Coordinator Katie James, UNHSI Student Worker

slade
Download Presentation

Penny Hatcher, Supervisor and Grant Director Yaoli Li, CDC EHDI Coordinator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Quality Assessment of Minnesota Newborn Hearing Screening Data: A Pilot Study National EHDI ConferenceMarch 26, 2007 Salt Lake City Penny Hatcher, Supervisor and Grant Director Yaoli Li, CDC EHDI Coordinator Nicole Brown, HRSA UNHSI Coordinator Katie James, UNHSI Student Worker Sarah Solarz, EHDI Student Worker Judy Punyko, MDH Epidemiologist Minnesota Department of Health (MDH) Community & Family Health Newborn & Child Screening Unit

  2. Faculty Disclosure Information In the past 12 months, I have not had a significant financial interest or other relationship with the manufacturer(s)of the product(s) or provider(s) of the service(s) that will be discussed in my presentation. This presentation will not include discussion of pharmaceutical or devices that have not been approved by the FDA or if you will be discussing unapproved or “off-label” uses of pharmaceuticals or devices. 2

  3. In compliance with the CDC guidelines for evaluating public health surveillance systems… Study Purpose • Assess the quality of Minnesota newborn hearing screening data • -Validity • -Reliability 3

  4. Methodology – Planning Phase • Develop partnership with • Vital Records (CHS) • Birth Defects Information System (EH) • Newborn Bloodspot Screening (PHL) • Newborn Hearing Screening • Data fields • Medical record abstraction form 4

  5. Medical Record Abstraction Form (Infants) 5

  6. Medical record abstraction form (7) BIRTH DATE From BC: 01/10/2005 mm dd yyyy *9-fill fields if missing (15) LEFT EAR SCREEN RESULTS, 1 MONTH 1 = Pass 2 = Fail 9 = Not screened/missing BS DATA 1 (11) MULTIPLE BIRTHS / BIRTH ORDER (a, b, etc.) 1 = Single Birth 2 = Twin ___ (a or b) 3 = Triplet ___ (a, b, or c) 4 = Quadruplet ___ (a, b, c, or d) 5 = Quintuplet ___ (a, b, c, d, or e) 6 = Other 9 = Unknown, not stated, unclassifiable BC DATA 2 a 6

  7. Methodology – Planning Phase • Select 20 MN hospitals with ≥15 births in 2005 • Hospitals rank-ordered by size (i.e. # births) • Every 5th hospital chosen • e.g. • Hospital Size • A 100 • startB 93 • C 90 • D 86 • E 82 • F 79 • G 70 7

  8. Methodology – Implementation • Two graduate student workers oriented by BDIS staff • Vital Records randomly selects mothers (n = 200) and their infants (n = 200) from birth certificates (Total N = 400) 8

  9. Methodology – Implementation • Letters sent to hospitals • Phone calls made to confirm appointments • List of 10 infant and 10 mother records faxed to each hospital 9

  10. Students hit the road… 10

  11. Methodology – Data Collection • At each hospital… • Collect and record information from medical records • Upon return to MDH… • Students double enter data 11

  12. Methodology – Data Analysis • Assess inter-rater reliability • How well do students’ data agree? • If discrepancies, determine which “answer” is correct • Create final (corrected) database 12

  13. Methodology – Data Analysis • Merge medical record (MR) data with hearing screening (HS) data 13

  14. Methodology – Data Analysis • Clean the merged dataset • e.g. Duplicate records • Use only records with “lab” or “loose” (i.e. from birth hospital) designation as data source. • NAMEBC NUMBERDOBSOURCE • Baby, Girl 2005-MN-0123456 1/1/2005 lab • Baby, Girl 2005-MN-0123456 1/1/2005 C1 refer • Baby, Girl 2005-MN-0123456 1/1/2005 refer 14

  15. Methodology – Data Analysis • Clean the merged dataset • e.g. Missing records • 5 infants • 2 died (no hearing screen done) • 1 transferred to NICU (no record of HS at birth hospital) • 2 with evidence of HS in medical record but were not in HS database 15

  16. Methodology – Data Analysis • Missing values • Recoded into a ‘no/unknown’ category • Weighting scheme 16

  17. Methodology – Data Analysis • Analysis of categorical (yes/no) variables included calculations of: • Sensitivity • Specificity • Positive predictive value 17

  18. Sensitivity Medical Record – Left Ear Results HS data – Left Ear Results Sensitivity = 160 / 167 = 95.8% 18

  19. Specificity Medical Record – Left Ear Results HS data – Left Ear Results Specificity = 20 / 33 = 60.6% 19

  20. Positive Predictive Value Medical Record – Left Ear Results HS data – Left Ear Results PVP = 160/173 = 92.5 20

  21. Results – Left Ear/Right Ear Screens 21

  22. Results – Left Ear/Right Ear Screens • % of infants with pass results in L and R ears: • ↑ sensitivity and ↓ specificity. -9/200 infants: L/R ear results in MR but missing in HS database. • 13/200 infants: L/R ear results in HS database but missing in MR. 22

  23. Results – Reasons For No Screen 23

  24. Results – Reasons for no screen • ↑ specificity • Variable sensitivity • Variable positive predictive value • In HS database, of 24/200 infants with missing L/R ear results… • Only 11 out of 24 with reason for why screen was missing or not done. 24

  25. Discrepancies between dates Infant Birth Dates Hearing Screen Dates 0 days 194 129 1-3 days 1 22 4-7 days 0 0 8-14 days 0 2 15-21 days 0 2 22+ days 0 3 Overall range -1 to 0 -63 to 366 Results – Birth and Screen Dates 25

  26. Results – Birth and Screen Dates MR and HS: • 3% of infant birth dates did not agree • 29% of HS dates did not agree • Of the 58/200 discrepant screen dates: • 22 missing in MR but available in the HS database • 5 missing in the HS database but available in MR 26

  27. Noteworthy Findings • Data fields with less frequent outcomes have lower validity: • Antibiotic use • Failed hearing screen results (left or right ear) 27

  28. Noteworthy Findings • Low to moderate agreement among the various reasons for no hearing screen • But… small numbers = reduced precision • Hospital-specific results 28

  29. Noteworthy Findings • Moderate agreement and high % missing among hearing screening dates • 29% overall disagreement • 17% missing in medical records • 11% missing in hearing screening database. • Some screen dates in HS database recorded as being prior to birth date 29

  30. Study Limitations: • Errors in abstraction process - Information recorded as “missing” by abstractors - Misinterpretation of language or results in medical record • Medical record as “gold standard” assumption 30

  31. Where do we go from here? • Implement safeguards in hearing screening database • Meet with hospitals • Modify study design 31

  32. Thanks You! Minnesota Contact Information: Penny Hatcher, MSN, DrPH penny.hatcher@health.state.mn.us 651-201-3744 Yaoli Li, MD, MS, MA-CCC.A yaoli.li@health.state.mn.us 651-201-3750 32

More Related