Refugee and Immigrant B Notification Program Evaluation—Idaho

1. Refugee and Immigrant B Notification Program Evaluation—Idaho Ellen Zager Hill, MS TB Program epidemiologist Idaho department of health & welfare

2. Idaho Geography (1) • Geographically large western state with a low population • 19.2 people per square mile (ranked 44th in US) • Population ≈ 1.58 million people • 37% of population is in the Boise metro area (Ada & Canyon Counties)

3. Idaho Geography (2) 44 counties divided into 7 multi-county regional public health districts 95%+ of refugees and immigrants settle in Boise (Ada County) and Twin Falls (Twin Falls County).

4. Background For a state with the size population that Idaho has, the state receives an unusually large number of refugees and immigrants 2 local public health districts receive essentially all of the refugees arriving in the state: PHD4 (90%+) and PHD5 (~10%) Immigrants arriving in Idaho are pretty evenly distributed; in SFY 2007–2009, 83 immigrants with TB notifications arrived

5. Evaluation Plan

6. Outcomes Activities Outputs Inputs Short term Long term OEFI receives notification from CDC Refugees and immigrants with a TB notification are medically evaluated for TB. Refugee/ Immigrant with a TB notification The LPHD will increase the proportions of completion of medical evaluations within 90 days. Reduce the number of foreign born TB cases. District receives notification from OEFI Every LPHD has a process for contacting, evaluating, and treating refugees and immigrants with TB notification. Contact Refugee or immigrant Initiate medical evaluation LPHD Increase the proportions of refugees/immigrants who started on LTBI treatment who complete treatment. Complete medical evaluation Refugees or immigrants treated who need to be treated. Start treatment for LTBI State TB Program Staff Data for grant progress report. Complete treatment for LTBI Private Provider Fill out CDC TB Notification form Form turned into state TB program Logic Model

7. Evaluation Plan (1) Evaluation Goal Evaluation Team Baseline analysis of medical screen activities of refugees and immigrants arriving in Idaho with a TB notification • Lead Evaluator • Ellen Zager Hill – TB PEN Focal Point/TB Epi • Team Members • Christine Hahn – TB Controller/State Epi • Adele Smith – MPH student from ISU

8. Evaluation Plan (2)

9. Evaluation Plan (3)

10. Evaluation Plan (3) Data Sources Execution Needs Idaho Refugee & Immigrant Log (MS Excel spreadsheet) CDC’s Electronic Disease Notification System Interviews with PHD staff who coordinate TB activities (via phone) Time Survey questions Phone Software to map processes (MS Visio) (relatively) Clean TB notification data Data analysis tool (MS Excel and SAS)

11. Carrying Out theEvaluation Plan

12. Carrying Out the Evaluation Plan (1) Developed a set of questions to ask the PHD staff who carry out TB activities Developed a script to go with the questions

13. Carrying out the Evaluation Plan (2) A flow of TB notification data from the state-level perspective was developed (shown at left) Based on survey questions, flow charts of the evaluation and treatment processes were developed for each PHD

14. Carrying out the Evaluation Plan (3) Data Cleaning Data Analysis • Started with our Idaho Refugee & Immigrant Log • Took out all records that were not for TB notifications • Filled in what missing data we could from hard copy records and the EDN system • Initially done with MS Excel • Final analysis done with SAS • Benefit of using a program like SAS is that you have a record of exactly how the data was cleaned, what analysis was done, and how it was done.

15. Carrying out the Evaluation Plan (4) Some lessons learned… • Logs (especially those with no data standardization programmed in) are a difficult data source to analyze • (A lesson re-learned) small denominators can be a problem • Clearly document how you collect, clean, and analyze data • An evaluation report is like a lab report. • Your methods section may be your most important section. (Sometimes even more important than your results.) • SAS or other statistical analysis programs are a great way to document how you are cleaning and analyzing data

16. Evaluation Results

17. Evaluation Results (1) • Are the medical evaluations for refugees and immigrants with TB notifications done in a timely manner? • What is the proportion of refugees or immigrants that were evaluated within 90 days of arrival? Overall (SFY2007–2009) 67% were evaluated within 90 days of arrival.

18. Evaluation Results (2) • Are the PHDs sending notifications back to the OEFI in a timely manner? • What is the proportion of TB notification forms that are received within 120 days of receiving the notification? Overall 76% of TB notification forms were returned within 120 days of receiving the original notification.

19. Evaluation Results (3)

20. Evaluation Results (4) Extra analysis tied to differences in processes… *Excludes data from 1 PHD (the PHD with the most notifications) because we were unable to interview them.

21. Recommendations PHD-level State-level Ensure there is a system in place to track TB notifications Review process for coordinating medical evaluation on a semi-regular schedule Consult OEFI TB staff as needed when a problem related to medical evaluation arises Consider adding language to TB contracts stipulating a time frame for returning TB follow-up forms Update B Notifications chapter of Idaho TB manual Provide at least 1 NTIP-like report a year to each PHD that contains an analysis of their TB notification statistics Review data from calendar years 2009, 2010, and 2011 for improvement. If improvement is not seen, then consider re-evaluating this activity.

22. What has been done as a result of the evaluation This data reflects only notifications for which data has been received. Please note that data for all years are incomplete for various reasons. The improvements in data are mainly due to process changes at the PHD that receives 90+% of refugees in the state. (This PHD was not interviewed for the evaluation.) We are no longer using the Idaho Refugee and Immigrant log to monitor performance for this objective. We are using solely EDN data. We added language to contracts stipulating when TB follow-up data needs to be submitted. B Notifications chapter of TB manual has been reviewed an updated, but has not yet been finalized. Completion of medical evaluation statistics as of 1/31/2012: (calendar years)

23. Questions