Developing a Valid Surveillance Design (Or…Is there life after mandatory reporting?)

1. Developing a Valid Surveillance Design(Or…Is there life after mandatory reporting?) Ona Montgomery Amarillo VA Health Care System

2. Proposed Initial Mandatory Reporting Components • CL-BSI • NICU - birthweight stratified • ICU – stratified by type of ICU • Special Care Units – stratified by type of unit • SSI • CMS procedures • Stratified by NHSN index • ASC and general hospitals • HCA-RSV • Pedi hospitals • General hospitals with pedi units

3. Other Surveillance Drivers • Regulations and requirements • JCAHO • CMS • Corporate mandates • Internal strategic plans • Single acute care facility or part of larger system?

4. How do we develop a scientifically valid and rational surveillance design when so many aspects seem out of our control?(As well as so much competition for available personnel and other finite resources) Approach with an epidemiologic perspective.

5. Recommended Practices for Surveillance • Assess the population • Select the outcome or process to survey • Apply surveillance definitions • Collect surveillance data • Calculate rates & analyze surveillance findings • Apply risk stratification methods • Report and use surveillance information APIC Surveillance Initiative Working Group, 1998 (Under revision)

6. Key Epidemiologic Principle of Surveillance Focus on populations or individuals at risk for infection.

7. Healthcare Venues Acute care ICUs LTC Rehab LTAC Ambulatory Home care Patient Populations Elderly Trauma Diabetic Oncology Dialysis Spinal cord injury Other Important risk populations Organizational population assessment should be comprehensive

8. Epidemiologic Principle Focus on priority problems/improvement objectives • Greatest potential for improvement (high volume, high risk, high intervention impact) • Significant outcome • Organizational objectives • Cost-effectiveness • Risk management • Existence of standards and mandates

9. Surveillance Practice 1:Assess the Population • Obtain data to describe / understand YOUR patient population • Establish priorities for surveillance through your annual risk assessment • Which patients or staff in your organization are at increased risk for infection?

10. Surveillance Practice 2:Select Outcome or Process to Survey Consider... • Relative frequency of the event (high volume) • Cost or impact of a specific negative outcome (high risk) • Preventability (high intervention impact) • Cost-benefit • Customer needs (e.g. priorities set by facility plan, regulatory compliance) • Organizational mission / strategic goals • Available resources (administrative commitment) Cost-effectiveness

11. The Surveillance Process • Assessing populations for risk • Choosing problem or event to be studied • Often done simultaneously • Factor in mandatory reporting components

12. Outcome and Process Measures • Outcomes • Infections • Events • Processes • Known to be related to outcome • Key patient care steps • Bundles • Search of literature, opinions of experts, or practice standards may point to appropriate process indicators

13. Process Measures - Examples • Prophylactic antibiotic timing in ambulatory surgery • Antibiotic selection in treatment of outpatient pneumonia • Immunization rates in all settings - staff and patients • Adherence to disinfection processes in dialysis center • Compliance with bundles prevent ventilator pneumonia • Device utilization ratios - with or without device-associated infection rates in home care, LTC, ICUs • PPD skin testing compliance of patients and/or staff - in conjunction with outcome measure of PPD skin test conversions

14. Surveillance Practice 4:Collect Surveillance Data • Data collection methods must be systematic • Utilize available information systems (including electronic data) • The team approach is critical - negotiate for assistance • Provide surveillance personnel with standardized training • Intensity of surveillance must be maintained over time!

15. Surveillance Practice 5:Calculate Rates and Analyze Findings • Plan for data analysis. • Express surveillance information in numerical terms (I.e. incidence density, rates) • Turn surveillance “data” into “information” • Use statistical probability methods to determine whether observed differences in rates are meaningful (i.e. significance testing; p values) • Provide analysis and interpretation

16. Use scientifically sound, well understood methods in data collection and analysis • Calculate rates only when numbers are sufficient • Numerator analysis and reporting may be appropriate for infrequent events • Control chart theory has been applied to rate- and event-based data

18. Surveillance Practice 6:Apply Risk Stratification Methods Risk stratification: subdividing (stratifying) your surveillance population into groups at similar levels of infection risk prior to performing any analyses or comparisons. To ensure comparing “apples to apples”

19. Patient-specific Risk Score Total 0-3 points Wound class class III or IV 1 point ASA score 3, 4, 5 1 point Duration of surgery > cutpoint 1 point CDC NNIS Risk Index for SSI Surveillance

20. CDC NNIS Risk Stratification for High Risk Nursery (HRN) Surveillance Stratification by Birthweight Categories: • </= 500 grams • 501 - 1000 grams • 1001-1500 grams • 1501-2500 grams • >2500 grams

21. Surveillance Practice 7:Report and Use Surveillance Data • Plans for the distribution of surveillance information should be incorporated into the development of each surveillance objective • Provide feedback to clinicians! • Reporting should be timely • Present surveillance findings using graphs and easy-to-read tables

22. “Surveillance without action should be abandoned.” Present surveillance data in a manner to stimulate ideas for process improvement. Perform follow-up surveillance to monitor for improvement following changes (“close the loop”).

23. Compared to what? • Internal changes over time • External comparisons such as: • NNIS/NHSN • IHI publications • State reporting data

24. ** Number of central line-associated BSI Number of central line-days X 1000 Pooled means and percentiles of the distribution of device-associated infection rates, by type of ICU, NNIS, 1/95-6/03 Central line-associated BSI rate** Percentile No. of Central Line- Pooled 10% 25% 50% 75% 90% Type of ICU Units Days Mean (median) Coronary 114 363,976 4.2 0.0 1.9 4.2 5.8 8.4 Cardiothoracic 71 598,118 2.9 0.4 1.3 2.2 3.5 4.9 Medical 143 975,318 5.7 2.1 3.4 5.0 6.8 9.6 Medical-Surgical Major teaching 133 936,223 5.0 2.2 3.0 4.9 6.3 7.7 All others 187 1,295,477 3.7 0.0 1.8 3.3 5.0 6.8 Neurosurgical 52 180,581 4.8 0.0 2.5 4.1 6.5 9.0 Pediatric 79 428,104 7.3 0.7 3.8 5.9 8.8 11.5 Surgical 160 1,267,959 5.2 1.1 2.6 4.7 6.9 9.3 Trauma 28 178,179 7.8 2.5 5.2 6.6 10.0 12.3

25. Pooled means and percentiles of the distribution of device utilization ratios, by type of ICU, NNIS, 1/95-6/03 Central line utilization** Percentile No. of Pooled 10% 25% 50% 75% 90% Type of ICU Units Patient-Days Mean (median) Coronary 115 1,120,967 0.32 0.13 0.21 0.29 0.42 0.58 Cardiothoracic 71 751,547 0.80 0.57 0.70 0.82 0.91 0.95 Medical 143 1,905,674 0.51 0.30 0.37 0.52 0.64 0.75 Medical-Surgical Major teaching 133 1,688,840 0.55 0.35 0.45 0.55 0.64 0.73 All others 187 2,770,191 0.47 0.25 0.34 0.47 0.57 0.63 Neurosurgical 52 401,236 0.45 0.26 0.38 0.49 0.55 0.63 Pediatric 82 936,169 0.46 0.20 0.30 0.41 0.53 0.60 Surgical 160 1,958,691 0.65 0.44 0.55 0.67 0.76 0.86 Trauma 28 280,074 0.64 0.47 0.57 0.65 0.75 0.85 ** Number of central line-days Number of patient-days

26. Surgical Prophylaxis (% given < 1 hr. before incision)

27. Development of your Hospital Surveillance Plan • Plan must be written! • Reflect logical risk assessment • Address internal and external priorities • Describe each surveillance component • Populations • Outcomes of concern • Rationale (e.g., risk assessment rating) • Measurement/analysis • Interventions and reporting plan

28. Risk Rating Example

29. Plan Example

30. Key Principle • Include clinicians in all steps of planning and implementing surveillance plan. • Improved buy-in and participation • Non-acute settings -- technical staff • Examples • ICU - front line staff (bundles) • LTC – CNAs as well as nurses • Home health - family educators

31. Key Principle • Report data to clinicians who can influence patient care practices. • Important tie between surveillance and clinical care • Reinforces staff involvement • Allows completion of surveillance cycle

32. Fostering Effective Interventions • Use stakeholders and content experts • Effective? • Feasible? • Barriers? • ICPs should strive to facilitate, not dictate or implement • Document improvement activities undertaken by all participants

33. Follow-up • Continued surveillance for internal comparisons to measure improvement • Outcome monitoring - look for improved infection rates • Process monitoring - look for improved rates of compliance - important even without corresponding change in outcome rates

34. “A good surveillance system does not guarantee you will make the right decisions, but it reduces the chances of making the wrong ones.” Dr. Alexander Langmuir N Eng J Med 1963;268:182-92.

35. References and Resources • Guidance on Public Reporting of Healthcare-Associated Infections AJIC Feb. 2005 http://www.cdc.gov/ncidod/dhqp/pdf/hicpac/PublicReportingGuide.pdf • Lee TB, Baker OG, Lee JT, Scheckler WE, et al. Recommended Practice for Surveillance. American Journal of Infection Control 1998;26:277-288. http://www.apic.org/AM/Template.cfm?Section=Surveillance_Definitions_Reports_and_Recommendations&Template=/CM/ContentDisplay.cfm&ContentFileID=2710 • NHSN Member site (surveillance protocols, etc.): http://www.cdc.gov/ncidod/dhqp/nhsn_members.html • Gaynes R, Richards C, Edwards J, et al. Feeding Back Surveillance Data to Prevent Hospital-acquired Infections. Emerging Infectious Diseases Mar-Apr 2001 http://www.cdc.gov/ncidod/eid/vol7no2/pdfs/gaynes.pdf • Haley RW. Surveillance by objective: a new priority-directed approach to the control of nosocomial infections. Am J Infect Control 1985;13:78-89.