Session 3: Assessment & Evaluation

1. Lisa Jacques-Carroll, MSW NCIRD, CDC Session 3: Assessment & Evaluation

2. Learning Objectives • Describe key components of evaluation of a perinatal hepatitis B prevention program • describe different methods for assessing and evaluating a perinatal hepatitis B prevention program • identify several tools that are available

3. Assessment & Evaluation • Program evaluation vs. program monitoring or assessment • The focus of this session is on assessing performance measures, which is a component of program evaluation • For more information on program evaluation see:  www.cdc.gov/vaccines/programs/progeval/

4. Activities Required by CDC • Requirements for assessment of the perinatal hepatitis B include: • frequency of assessment • performance measures • target levels • For more information: http://www.cdc.gov/vaccines/vac-gen/policies/ ipom/downloads/chp-05-perinatal-hepb.pdf

5. Overview • Laboratory reporting audits • Hospital record reviews • Hospital policy surveys • Expected births to HBsAg+ women • Assessing case management • Monitoring birth dose

6. Laboratory Reporting Audits • Use to assess: • completeness of reporting of HBsAg+ results • timeliness of reporting • Include delivery hospital labs • Collaborate with communicable disease and state laboratory staff to help plan and execute reviews

7. Prioritize Labs to Evaluate • Priority labs: • labs serving high-morbidity areas or populations • labs reporting large volume of hepatitis serology • labs serving prenatal clinics • delivery hospital labs • How often? • ideally, once/year for priority labs • every 2–3 years for others • minimally every 5 years

8. Laboratory Audit Tool • A CDC tool available at: http://www.cdc.gov/ncidod/diseases/hepatitis/resource/LabReportingAssessment.doc

9. Completeness of Lab Reporting • Pick a time period (at least 3 months) • Calculate: Total # HBsAg+ test results reported Total # specimens that were HBsAg+ Proportion of HBsAg+ tests reported, or completeness of reporting =

10. Timeliness of Lab Reporting

11. Timeliness of Lab Reporting - cont’d Sum of column “# days between serology and report” Total # of serologies evaluated Mean reporting time for the laboratory, or timeliness of reporting = 25 4 = 6.25 days e.g. from the previous slide Time period for reporting is typically set by state statute

12. Tips for Working with Laboratories • Contact other areas of health department to see if activity is already being done • See if an existing lab report can be used for the audit—ensure it is source data • When making contact with labs, include: • lab director • technician • IT staff

13. Hospital Record Reviews • Use to assess: • maternal HBsAg screening • birth dose administration • post-exposure prophylaxis (PEP) for infants born to HBsAg+ and unknown-status women • Audit paired maternal and infant records • Partner with health dept programs to collect other perinatal data • rubella antibody • HIV • group B strep • early hearing • syphilis

14. Determining Sample Size • Tool available in Guide to Life (Table 2.1) • http://www.cdc.gov/ncidod/diseases/ • hepatitis/resource/perinatalhepB.htm

15. Using the Guide to Life table • Determine hospital’s annual number of deliveries • Decide where it falls in the “Birth Cohort Size” category • Estimate HBsAg screening and hepatitis B birth dose coverage for the hospital • Use the lower coverage to determine the sample size

16. Sample Size Example Hospital X had 648 deliveries last year The expected HBsAg screening rate is 95% The state National Immunization Survey (NIS) birth dose rate is 65% Using the 65% coverage level, the sample size for Hospital X would be 111 records

17. Hospital Data to Collect • Delivery date/time • Mother’s HBsAg test date/result/type in: • maternal record • infant record • Infant hepatitis B vaccination (yes/no) • Date/time of vaccination • Infant HBIG administration (yes/no)

18. Hospital Data to Collect- cont’d • Date/time of HBIG • Other variables as appropriate

19. Hospital Policy Surveys • Use to assess whether hospitals have written policies/standing orders in place: • universal birth dose of hep B • HBsAg screening of pregnant women • PEP to infants born to HBsAg+ and unknown-status women • documentation of maternal HBsAg status, birth dose, and HBIG (if appropriate) in medical records

20. Conducting Policy Surveys • Mail survey • Telephone survey • In-person survey during hospital medical record reviews Keep in mind, policies do not always match practice, so medical record reviews are essential

21. Expected Births to HBsAg+ Women • CDC expected birth estimates are a benchmark for programs • CDC provides estimates for states and some cities • Estimates are calculated using: • birth data • prevalence of chronic HBV infection by race/ethnicity

22. Expected Births- cont’d • Nationally, about 24,000 births expected annually, only 50% identified in 2005 • CDC encourages programs to improve estimates of births to HBsAg+ women • More information in presentation at: http://www.cdc.gov/ncidod/diseases/hepatitis/resource/presentations/2007-04PeriGrantees/PDFs/May1-p0130-1LJ_2007PeriCoordMtg-ExpectedBirths.pdf

23. Assessing Case Management • Number of infants born to HBsAg+ women • Proportion of infants receiving • hep B vaccine & HBIG within 12 hours of birth • on-time completion of hep B vaccine series • post-vaccination testing • Reasons for cases lost to follow-up • Proportion of screened and vaccinated household and sexual contacts

24. Assessing Case Mgmt - cont’d • Monitor case management outcomes: • compare program-wide rates (entire state or city) to national rates (CDC peritable) • regionally • locally (county level) • by case worker • quarterly or monthly • over time (trends)

25. Monitoring Birth Dose • National Immunization Survey (NIS) data • measures hepatitis B birth dose rates • Hospital medical record reviews • Immunization Information Systems (IIS) • hep B vaccine birth certificate data automatically uploaded into IIS • calculate birth dose coverage for cities, regions, and individual hospitals

26. Continuous Monitoring • Some aspects of the program require continuous monitoring • case management outcomes • laboratory reporting • Plan ahead for assessments your program will conduct over the next year/s

27. Using Your Results • Determine future direction and focus of program efforts • Provide feedback to: • hospitals • laboratories • providers • health departments • case workers