Birth data quality

1. Birth data quality VITAL RECORDS: A CULTURE OF QUALITY NAPHSIS Annual Meeting | Seattle | June 8-11, 2014

2. the birth data quality workgroup An Update

3. Birth Data quality workgroupHospital reports/ engaging hospitals Subgroup Karyn Backus (Chair) (CT) Colleen Fontana (Chair) (RI) Sukhjeet Ahuja (NAPHSIS) Mary Chase (CO) Greg Crawford (KS) Melissa Gambatese (NYC) Jean Hreczan(DE) Andrew Jessen(AK) David Justice (NCHS) Ann Madsen Straight (NYC) Joyce Martin (NCHS) Judy Nagy (OH) Sharon Pagnano(MA) Phyllis Reed (WA) Shae Sutton (SC) Marie Thoma (NCHS) Elaine Tretter (MD) Louise Wishart(DE)

4. Charge • Birth Data Quality Workgroup mission was to address identified issues with birth data quality. • Subgroup 1: Hospital Reports • Charge: Recommend a process (metrics, means of communication, actions) for vital records offices to use to provide data to hospitals to help them improve their reporting. • Goal 1: Survey all jurisdictions to learn their current activities

5. Charge • Subgroup 2: Engaging Hospitals • Charge: To develop approaches to engaging hospitals to improve systems and procedures for data gathering. • Step 1: Develop an outline for engaging hospitals • In October 2013, subgroups 1 and 2 were combined as charges were overlapping.

6. Survey results

7. WHY DID WE DO THE SURVEY? The survey is a starting point; a tool to gauge where we are and who is doing what. • Are all jurisdictions actively engaged in Evaluating and Ensuring Data Quality (EEDQ) from the birth facilities? • Is communication with hospitals on EEDQ issues regularly utilized as a means for improving quality? • Does communication with hospitals make a difference?

8. WHY DID WE DO THE SURVEY? The survey is a starting point; a tool to gauge where we are and who is doing what. (cont.) • What else are jurisdictions doing to improve the data quality from hospitals? (types of metrics, types of awareness efforts) • Do jurisdictions want to do more or are they satisfied with where they are? • What other barriers are there to EEDQ (organizational dynamics, staffing, failure to utilize standard tools, etc.)

9. SURVEY PARTICIPATION • First administered in March 2013 • 41/52 (78.8%) – United States • 2/5 (40.0%) – Territories • In October 2013, non-responding jurisdictions contacted by email with opportunity to participate • 5 additional U.S. jurisdictions responded

10. SURVEY PARTICIPATION • Final participation rates: • 46/52 (88.4%) – United States; representing 83% of US Births • 2/5 (40.0%) – Territories All jurisdictions Did not participate 15.8% Participated 84.2%

11. FINDINGS & RECOMMENDATIONSJURISDICITONAL DYNAMICS Encourage Improved Collaboration within the Jurisdiction • Findings: • Majority maintain registration and statistics staffs in same unit • Majority indicate room for improvement to maximize EEDQ • More than half reported being understaffed for EEDQ activities • Nearly all report desire to do more extensive EEDQ

12. FINDINGS & RECOMMENDATIONSJURISDICITONAL DYNAMICS Encourage Improved Collaboration within the Jurisdiction • Recommendations: • Jurisdictions should improve collaboration between staffs • Next steps: • Identify jurisdictional successes with or impediments to collaboration • Offer recommendations for improving or enhancing the relationship between the two staffs

13. FINDINGS & RECOMMENDATIONSSTANDARDIZATION NOT UNIVERSAL Increase Jurisdictional Compliance with Established Standards • Findings: (revised only)

14. FINDINGS & RECOMMENDATIONSSTANDARDIZATION NOT UNIVERSAL Increase Jurisdictional Compliance with Established Standards • Recommendations: • All jurisdictions should develop standardized worksheets based on NCHS standard • Jurisdictions should mandate use by birth facilities • Next steps: • Clarification from NAPHSIS/NCHS regarding what constitutes compliance • Explore why jurisdictions did not adopt standards • Consider improving the standards based on identified issues

15. FINDINGS & RECOMMENDATIONSSTANDARDIZATION NOT UNIVERSAL Increase Jurisdictional Compliance with Established Standards • Findings (all jurisdictions): • EBRS vary in their forward-facing quality control measures • Missings versus unknowns • Logic checks, soft edits, hard edits • Jurisdictional interpretation of completeness varied • Some reported that EBRS was sufficient to ensure data quality • Recommendations: • Jurisdictions should confirm that systems are compliant with standards

16. FINDINGS & RECOMMENDATIONSSTANDARDIZATION NOT UNIVERSAL Increase Jurisdictional Compliance with Established Standards • Next steps: • Clarify whether missing versus unknown is a QA concern as it pertains to record completeness • Review existing standards for data collection systems and document rationale for why various standards are imposed • Share review with jurisdictions so they can better evaluate their systems

17. FINDINGS & RECOMMENDATIONSDATA QUALITY EVALUATIONS Increase Data Quality Evaluations • Finding 1: • Range in jurisdictional utilization of data quality analyses • Recommendations: • Learn about the various metrics available for assessing quality • Self-evaluate to expose data validity issues • Next steps: • Develop ways to educate jurisdictions about data quality constructs • Collaborate with NCHS to provide additional data quality reports

18. FINDINGS & RECOMMENDATIONSDATA QUALITY EVALUATIONS Increase Data Quality Evaluations • Finding 2: • Types of QA metrics utilized varies • Frequency and timing of QA reports varies • Completeness: 52% outside EBRS • Logic checks: 61% outside EBRS • Audits: 6% regularly, 17% rarely or as needed • Other QA: 78% perform at least one, 17% perform all three

19. FINDINGS & RECOMMENDATIONSDATA QUALITY EVALUATIONS Increase Data Quality Evaluations • Recommendations: • Move closer to near real-time evaluations and away from year-end • Use multiple metrics to detect quality issues • Look for alternative resources for QA, given staffing limitations • Hospital self-audits, collaborate with other programs, funding for targeted QA, linkage with hospital discharges

20. FINDINGS & RECOMMENDATIONSDATA QUALITY EVALUATIONS Increase Data Quality Evaluations • Next steps: • Determine which metrics are efficient and effective at measuring quality and recommend these as a best practice • Develop standardized analytical programs for recommended metrics (69% prefer SAS) • Investigate the development of NCHS-based hospital reports

21. FINDINGS & RECOMMENDATIONSProvide feedback Advocate for Data Quality • Findings 1: • Jurisdictions that provided feedback realized improvement. • Some jurisdictions have no efforts in place to advocate for birth data quality; many have only a few targeted efforts.

22. FINDINGS & RECOMMENDATIONSprovide feedback Advocate for Data Quality • Recommendations: • Improve communication with hospital partners about performance. • Increase education and awareness about the merit of data quality and common issues. • Provide trainings and newsletters. • Educate hospital staff about the public health value of birth certificate data. Educate non-hospital staff about data quality initiatives and data limitations. • Publish reports about performance to increase transparency.

23. FINDINGS & RECOMMENDATIONSprovide feedback Advocate for Data Quality • Next Steps: • Develop a set of best practices for communicating with hospitals about ongoing performance. • Develop educational materials (newsletters, letters, reports) for hospital staff and non-hospital stakeholders. • Evaluate expressed concerns over publishing hospital-level reports (e.g., confidentiality, misinterpretation).

24. FINDINGS & RECOMMENDATIONSprovide feedback Advocate for Data Quality • Finding 2: • The hospital administration was rarely identified as recipients of performance reports. • Recommendation: • Engage upper-level clinicians and hospital administrators in the merit of providing quality birth data.

25. FINDINGS & RECOMMENDATIONSprovide feedback Advocate for Data Quality • Next steps: • Pursue effective avenues for communicating with higher-level hospital administration. • The subgroup will transition into the “Engaging Hospitals” charge.

26. SUMMARY • The national survey provided information that was not previously available. • Serves as a spring board for initiatives at the jurisdictional and birth facility level. • Standardization • Education • Communication • Supporting the overall initiative to develop a national model for improving birth data quality.

27. CONCLUSIONS 91% of jurisdictions want to do more quality review. Our subgroup is striving to provide tools to all jurisdictions to reach that goal. Thank you to all jurisdictions who participated in the survey. You are helping us go fromGoodto GREAT!

28. Birth Data quality workgroupPrenatal Care ItemsSubgroup Sukhjeet Ahuja (NAPHSIS) Karyn Backus (CT) Bruce Cohen (MA) Isabelle Horon (Chair) (MD) Renata Howland (NYC) Michelle Osterman (NCHS) Elaine Tretter (MD)

29. Charge • To assess the quality of prenatal care data items collected on the U.S. Standard Certificate of Live Birth; and • Recommend changes for improvement

30. LAST YEAR • Background • Data uses • History of collecting PNC data • Data quality • Barriers to the collection of accurate data • Survey of jurisdictions

31. This Year • Further review of survey findings and follow up with selected jurisdictions • Review of Guide to Completing the Certificate of Live Birth • Survey of MCH Experts • Development of Recommendations

32. PNC DATA ITEMS • Date of first prenatal care visit • Date of last prenatal care visit • Number of prenatal care visits

33. SURVEY • Last year—data collection • This year—telephone follow up with 11 jurisdictions • Efforts in place • Effectiveness of efforts

34. REVIEW OF GUIDE TO COMPLETING THE CERTIFICATE OF Live Birth • Prepared by NCHS(with assistance from NAPHSIS, birth information specialists, and clinical experts) • Purpose—to assist facility birth registrars in providing complete, accurate data • Contains: • Definitions • Instructions • Sources of information • Key words/abbreviations

35. SURVEY OF MATERNAL AND CHILD HEALTH EXPERTS • Queried MCHdata collection and research experts concerning “date of last prenatal care visit”

36. Recommendation 1Improve data collection • Include edits/edit rules in EBRS • Require facilities to obtain data from PNC records • Encourage PNC data collection from providers at 35+ weeks, with update following delivery • Consider preparing a comprehensive worksheet for PNC providers • Recommend that hospitals with EHRs store paper PNC records in central location

37. Recommendation 2improve training • Visit facilities routinely • Provide training for new birth registrars • Ensure facilities have (and use) Guidelines • Promote use of eLearning training

38. Recommendation 3improve communication with hospital staff and pnc providers • Hold annual conference for birth registrars • Recognize good performance • Prepare newsletter • Identify data users and solicit assistance • Communicate with PNC providers re: need for accurate data • Assist birth registrars to find solutions to problems

39. Recommendation 4review data quality and take action • Assess data quality regularly and contact facilities for corrected data ASAP • Provide facilities with completeness/accuracy reports • Provide comparison data to encourage improvement • Concentrate on poorest performing facilities • Recommend strategies for improvement • Follow changes in trends • Conduct simple analyses to identify hospitals providing questionable data

40. Source: Connecticut Department of Public Health

41. Source: Connecticut Department of Public Health

42. Source: Connecticut Department of Public Health

43. Source: Connecticut Department of Public Health

44. Recommendation 5audit hospital medical records • Conduct routinely • Review both mother and newborn records • Meet with hospital staff • Prepare detailed analysis of findings • Point out areas where improvement is needed • Provide instructions for improvement • Provide positive as well as negative findings • Provide findings to high level hospital staff • Conduct follow up audit in 6 months if facility is performing poorly. • Encourage hospitals to do internal audits • Look for outside resources to support audit

45. Recommendation 6Raise awareness of guidelines • Raise awareness • Encourage use

46. Recommendation 7certificate change • Drop “Date of Last Prenatal Care Visit” from the U.S. Standard Certificate of Live Birth • No body of literature or significant research findings describing use of this data item • Recorded inconsistently across facilities and jurisdictions • Incomplete • Little analytic, research or practical utility

47. Recommendation 8focus on improving “Number of Prenatal care Visits” • Establish clear definition • Enlist assistance from clinicians and MCH experts • Test to ensure that new definition improves data quality • If data quality cannot be improved, reevaluate whether item should remain on certificate

49. Birth Data quality workgroup Learning CreationSubgroup Sally Almond (Chair) (MN) Marie Aschliman (UT) Karyn Backus (CT) Lucy England (CDC) Colleen Fontana (RI) Melissa Gambatese (NYC) Saeed Hamdan (CDC) Catherine Haralson (TN) Kerry Lionadh (OR) Joyce Martin (NCHS) TJ Mathews (NCHS) Carol Moyer (KS) Marie Thoma (Chair) (NCHS) Elaine Tretter (MD) e

50. Why create learning? Birth record timeliness is much improved e