Best Practices for Data Submission and Analysis

1. Best Practices for Data Submission and Analysis Linda R. Greene, RN, MPS,CIC,FAPIC Manager, Infection Prevention UR Highland Hospital Rochester, NY linda_greene@urmc.rochester.edu

2. Objectives • Discuss the basics of Data Collection and Analysis • Describe basic infection measures used for reporting purposes • Review the role on NHSN in national data reporting • Identify how data can be used to drive improvement

3. Comparison Data Data for Action

4. Definition of Surveillance “ A comprehensive method of measuring outcomes and related processes of care, analyzing the data and providing information to the healthcare team to assist in improving those outcomes”

5. Standard Definitions • Definitions provide a method to objectively measure the intended outcome • The CDC National Healthcare system is considered the gold standard for surveillance systems in the US • IPs should review, learn and apply these defintions

6. Surveillance Essentials Know protocol/criteria Apply consistently Failure to report events: • Decrease usefulness of comparative data • Unfair comparisons between facilities • Possible validation discrepancies • Potential impact of CMS, scores and reimbursement

7. Basic Statistical Measures Rates, Ratios and proportions are used to measure the occurrence and risk of an event in a specific population over a given time period. Formula X Y 10 “n” Rates: Numerator and denominator represent 2 groups compared “10n” is used to covert to a whole number

8. Rates Most Commonly seen as incidence rate: Occurrence of new cases or events in a specific location over a defined period of time: • Device infection rates- N – usually per 1,000 line or UC Days • Surgical site infection rates- per 100 cases ( percentage) • Disease infection rates ( MRSA, C. difficile) usually per 10,000 patient days

9. Prevalence Rates • Prevalence is a statistical concept referring to the number of cases of a disease that are present in a particular population at a given time, whereas incidence refers to the number of new cases that develop in a given period of time. Pros: quick and easy. Can be performed on a particular day of the month (i.e. pressure ulcers) or in response to a situation Cons: Not useful for device infections , SSI, ETC.

10. Example :Hospital Onset C. difficile Infections

11. Rate CAUTI by Quarter ICU

12. Questions Why do we use UC days rather than patient days ? Why do we use 10,000 days for C. difficile rather than 1,000 ? Why would SSI rate be per 100 ?

13. Problem with rates • Not precise • Not easy to covert to single number • Risk factors and patient population may be difficult • May be helpful for internal monitoring over time

14. Ratios and Proportions A ratio is a fraction of a value in the numerator(X) that may not be included in the denominator(Y) Example : Device utilization ratio What proportion of patients on a given unit had a urinary catheter?

15. Attack Rate Incidence proportion Generally used to describe the frequency of cases during an outbreak i.e. 11 of 46 people at a luncheon developed a GI illness within 1 hour 11/46x 100 = 23.9%

16. NHSN What is NHSN? CDC’s National Healthcare Safety Network is the nation’s most widely used healthcare-associated infection (HAI) tracking system. NHSN provides facilities, states, regions, and the nation with data needed to identify problem areas, measure progress of prevention efforts, and ultimately eliminate healthcare-associated infections

17. Public Reporting and comparative data

19. Surveillance Definitions https://www.cdc.gov/nhsn/pdfs/cms/cms-reporting-requirements.pdf Important to understand these definitions Correct application is essential

20. Why Analyze? • Provide feedback to internal stakeholders • Analyzing HAI data can help facilitate internal validation activities • Reports can help inform prioritization and success of prevention activities. • Data entered into NHSN may be used by: CDC, CMS, your state health department*, your corporation*, special study groups*, etc. • At the end of the day, these are YOUR data – you shouldknow your data better than anyone else.

21. Metrics and Reports Don’t limit yourself! A number of different types of reports are helpful in analyzing your data… • Line Lists • Frequency Tables • Charts/graphical reports • Rate Tables • Standardized Infection Ratios (SIRs) • Descriptive statistics (e.g., mean, median, mode, distribution, outliers, etc.)

22. Metrics and Reports Line Lists • Allow for a record-level review of data • Helpful in pinpointing issues in data validity/quality • Can help to inform rates or other summarized measures • Can help in the identification of any trends • Can be used for SSI postdischarge surveillance efforts

23. When to Analyze Develop a timeline to regularly enter, and analyze your hospital’s data • Consider a timeline that would allow for timely feedback and interventions, if necessary • Example: Monthly review of rates and event-level details

24. Checklist • Before you begin analyzing, ask yourself these questions: • What data are you analyzing? • What is the time period of interest? • Why are you analyzing these data? • Who is the audience?

25. Standardized Infection Ratio Observed # of HAIs SIR = ------------------------------------------------- Expected (Predicted) # of HAIs Observed # of HAIs – the number of events that you enter into NHSN Expected or predicted # of HAIs – comes from national baseline data ( 2015) When the SIR = 1, then the number of observed = the number expected.

26. SIR • The SIR is called an indirect summary statistic • Simply put it compares the organization’s HAI experience to a pooled mean of a baseline period known as the referent period. • 2015 Baseline

27. Why the SIR? • The SIR is a risk-adjusted summary measure that compares the number of events of a given type observed/identified in a facility with the number expected (or predicted) of that type. • The number of expected events is calculated based on the types of data the hospital has reported and the NHSN baseline data. • The SIR uses past data (i.e., baseline) in order to predict future incidence. • The SIR is scalable and can be easier to interpret when assessing incidence at an overall level. • Statistical evidence is provided in the form of p-values and 95% confidence intervals to indicate if the hospital’s experience is any different than what is predicted based on the national data

28. Pros and Cons of the SIR

29. Lets look at statistical significance Standard deviation – variability around the mean or average P value – usually .05 is considered statistically significant • When taken by themselves, p-values really just provide the answer of whether or not we are going to consider a difference significant. They don’t indicate what direction the difference was in and don’t provide a feeling for how solid that answer is. To achieve that we need to look at the confidence intervals. • A confidence interval is the range of values that we are confident include the true value that we are trying to measure. In this case, when our alpha was set to .05 and we’re willing to risk a false positive 5% of the time, our confidence interval is a 95% confidence interval and represents the range of values we are 95% confident contain the true SIR for the facility.

30. Standardized Infection Ratios • Null hypothesis: • Your infection rate and the benchmark infection rate are the same. • Alternative hypothesis: • Your infection rate and the benchmark infection rate are different. Same Better Worse 0 1 2 SIR

31. Interpretation of the SIR • SIR = 1 • The number of infections is around what would be predicted . • SIR >1 • The number of infections is higher than predicted (worse). • SIR <1 • The number of infections is lower than predicted (better).

32. Standardized Infection Ratios α = 0.05 Same Better Worse 0 1 2 SIR

33. Standardized Infection Ratios α = 0.05 0 1 2 3 5 7 4 6 SIR

34. Standardized Infection Ratios • Null hypothesis: • Your infection rate and the benchmark infection rate are the same. • Alternative hypothesis: • Your infection rate and the benchmark infection rate are different. Same Better Worse 0 1 2 SIR

35. Interpretation of the SIR • SIR = 1 • The number of infections is around what would be predicted . • SIR >1 • The number of infections is higher than predicted (worse). • SIR <1 • The number of infections is lower than predicted (better).

36. Standardized Infection Ratios α = 0.05 Same Better Worse 0 1 2 SIR

37. Standardized Infection Ratios α = 0.05 0 1 2 3 5 7 4 6 SIR

38. Targeted Assessment for Prevention (TAP) Strategy

39. Why Use TAP for Data Reports? Use of a Prioritization Metric • Allows for calculation of the number of infections that must be prevented to meet an HAI reduction target • Identifies and prioritizes facilities or locations within facilities where the largest reductions can be achieved • Provides a focused approach to prevention • Promotes a standardized assessment of practice gaps that may be contributing to higher HAIs

41. TAP Strategy Target → Assess → Implement • Target facilities using TAP Report function available in NHSN • Assess gaps in infection prevention in targeted facilities/units using Facility Assessment Tools • Implement interventions to address the gaps in infection prevention using Implementation Guidance

42. Where to begin? Start with important terms: • Cumulative Attributable Difference (CAD) • A measure to target prevention to reach HAI goals • CAD = the number of infection to prevent in order to reach the SIR Goal

43. Example

45. Example

49. Practical Application

50. Advantages • Can be used in annual risk assessment • Helps care providers have realistic goals • Prioritize units at the facility level