1 / 64

When Is Quality Improvement Research?

When Is Quality Improvement Research?. Christopher J. Crnich, MD MS November 15, 2008. Outline. Hierarchy of evidence Misconceptions about the hierarchy Evidence of QI = research Acute care Long-term care Potential limitations of QI research

wells
Download Presentation

When Is Quality Improvement Research?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. When Is Quality Improvement Research? Christopher J. Crnich, MD MS November 15, 2008

  2. Outline • Hierarchy of evidence • Misconceptions about the hierarchy • Evidence of QI = research • Acute care • Long-term care • Potential limitations of QI research • Improving methodological quality of QI research

  3. A Short History of Medicine 2000 BC “Here, eat this root.” 1000 AD “That root is heathen. Here, say this prayer.” 1850 AD “The prayer is superstition. Here, drink this potion.” 1920 AD “That potion is snake oil. Here, swallow this pill.” 1945 AD “That pill is ineffective. Here, take this penicillin.” 1955 AD “Oops… bugs mutated. Here, take this tetracycline.” 1960-1999 AD “Thirty-nine more “oops”… Here, take this more powerful antibiotic.” ~2000 AD “The bugs have won! Here, eat this root.” Anonymous, World Health Report on Infectious Diseases 2000

  4. Hierarchy of Research Evidence

  5. Results from Observational Studies of Combined Hormone Therapy and from the Women's Health Initiative and the Heart and Estrogen/Progestin Replacement Study Grodstein, F. et al. N Engl J Med 2003;348:645-650

  6. Hierarchy of Research Evidence Quality Improvement Projects?

  7. Hierarchy on Shaky Ground Benson et al. N Engl J Med 2000; 342: 1878-1886

  8. The Tyranny of the RCT • "the claims for the RCT have been greatly, indeed preposterously overstated. The truth of the matter is that the RCT is one of many ways of generating information, of validating hypotheses. The proponents of the RCT, however, have elevated what is in theory a frequent (though by no means universal) advantage of degree into a gulf as sharp as that between the kosher and the non-kosher.” • Charles Fried

  9. Weaknesses of RCTs • Cannot be used to assess effects of potentially hazardous exposures • Traditionally examine efficacy not effectiveness • Difficulty with context • Have difficulty analyzing complex interventions

  10. Are RCTs the only way to learn? • “The difference between the RCT and the observational, retrospective study is not the difference between good and bad science, truth or falsity, but a difference between varying degrees of confidence.” • Charles Fried

  11. Traditional View of QI • Focused on implementing proven intervention in the real world (effectiveness) • Primary objective is to improve patient care, gain of knowledge is secondary • Local focus (not generalizeable) • Minimal statistical analysis • Not a viable form of academic productivity

  12. Can QI be Research? • It depends • From IRB perspective, no • From academic advancement perspective, yes • Like RCTs and causal epidemiologic studies QI (when done right) can be used to: • Generate new knowledge • Generalize that knowledge to other settings • Increasing methodological rigor of QI has been the key to academic acceptance

  13. Examples of QI Research

  14. NOSOCOMIAL BSI:IVD-RELATED BSIs • ~200 million IVDs/year in U.S. • ~0.2-5% risk of IVD-related BSI • Attributable mortality ~ 5% • Incremental costs $12,000/case • ~500,000 IVD-related BSIs in U.S. each year, ~25,000 deaths

  15. O’Grady et al. Clin Infect Dis 2002; 35(11): 1287-1301

  16. Guideline Compliance • Education program (52%) • Routine exchange of CVCs (25%) • Appropriate hand hygiene (17 – 60%) • Insertion in subclavian vein (31 – 60%) • Maximal sterile barriers during insertion (28 - 58%) • Chlorhexidine for insertion site preparation Alonso-Echanove et al. Infect Control Hosp Epidemiol 2003 Braun et al. Infect Control Hosp Epidemiol 2003 Coopersmith et al. Arch Intern Med 2004 Warren et al. Infect Control Hosp Epidemiol 2006

  17. QI Project to Reduce CVC-associated BSIs P < 0.0001 P < 0.0001 P < 0.0001 P = 0.04 Eggiman et al. Lancet 2000; 355(9218): 1864-8

  18. Intervention 1: Education of staff • Intervention 2: Creation of CVC insertion cart • Intervention 3: Asking if CVC can be removed daily • Intervention 4: Nurse completed checklist • Intervention 5: Empowerment of nurses Crit Care Med 2004; 32(10): 2014-20

  19. Crit Care Med 2004; 32(10): 2014-20

  20. Program Expansion: The Michigan Keystone Project Pronovost et al. N Engl J Med 2006; 355(26): 2725-32

  21. Program Expansion: The Michigan Keystone Project Pronovost et al. N Engl J Med 2006; 355(26): 2725-32

  22. Antimicrobial Use in LTCFs • Accounts for 20-40% of the medications used in LTCFs • 3-13% of residents are receiving antimicrobials at any time • 40-70% of residents will receive at least one antibiotic during 6 months of follow-up • 25-75% of antibiotics given for inappropriate indications

  23. Antimicrobial Use in LTCF Mylotte, AJIC 1999; 27:10-19

  24. Antimicrobial Use in LTCF Mylotte, AJIC 1999; 27:10-19

  25. Antimicrobial Use in LTCF Mylotte, AJIC 1999; 27:10-19

  26. Antimicrobial Use in LTCFs Mylotte, AJIC 1999; 27:10-19

  27. Phase 1: Antibiotic utilization review • Phase 2: Physician contact and guideline development • Phase 3: Intervention deployment • Pocket cards with Abx use guidelines • Caregiver lectures • Weekly antibiotic rounds • Targeted feedback to outliers J Am Geriatr Soc 2004; 52(1): 112-6

  28. J Am Geriatr Soc 2004; 52(1): 112-6 P < 0.001 P = 0.08

  29. J Am Geriatr Soc 2007; 55(8): 1236-42 • Point prevalence study of antibiotic resistance (1998) • Development of antimicrobial utilization methods (2000) • Teaching and guideline intervention • Targeted internists and nursing leaders • Case-based education • Dissemination of algorithms and guidelines

  30. J Am Geriatr Soc 2007; 55(8): 1236-42 -12.1%, NS -29.7%, P < 0.001 -25.9%, P = 0.06 -22.2%, NS Antibiotic-resistant infections (per 1,000-days)  25%

  31. Limitations of QI Research • Internal validity problems • Construct validity problems • Statistical validity problems • External validity problems

  32. Internal Validity Problems • Confounding factors, history • Trends/maturation effects • Seasonal effects • Detection bias • Attrition bias • Selection bias • Regression to the mean effects

  33. Internal Validity Problems • Confounding factors, history • Trends/maturation effects • Seasonal effects • Detection bias • Attrition bias • Selection bias • Regression to the mean effects

  34. Confounding • Problem • Many features of care can change over time (intentional and unintentional) • Reduces ability to infer causal benefit (or lack thereof) of intervention • E.g., MRSA control programs • Potential solutions • Make inventory of potential care practices that can influence outcome of interest • Seek to measure major confounders along with intervention • Adjust for confounding in analyses • When not feasible, fess up in your limitations section

  35. Determinants of MRSA Control Amox/Clav 3GCP Alch HH Alch Wipes Macrolide Fluoroquinolone ASC Admit MRSA Aldeyab et al. J Antimicrob Chemother 2008; 62(3): 593-600

  36. Confounding • Problem • Many features of care can change over time (intentional and unintentional) • Reduces ability to infer causal benefit (or lack thereof) of intervention • E.g., MRSA control programs • Potential solutions • Make inventory of potential care practices that can influence outcome of interest • Seek to measure major confounders along with intervention • Adjust for confounding in analyses • When not feasible, fess up in your limitations section

  37. Internal Validity Problems • Confounding factors, history • Trends/maturation effects • Seasonal effects • Detection bias • Attrition bias • Selection bias • Regression to the mean effects

  38. J Am Geriatr Soc 2007; 55(8): 1236-42 Δ, P = ???

  39. J Am Geriatr Soc 2007; 55(8): 1236-42

  40. J Am Geriatr Soc 2007; 55(8): 1236-42 Δ, P = ???

  41. Dealing with Maturation Effects • Multiple data points • Utilize time series statistical methods for analyses • Segmented regression • ARIMA (autoregressive integrated moving average) • Present data visually

  42. Internal Validity Problems • Confounding factors, history • Trends/maturation effects • Seasonal effects • Detection bias • Attrition bias • Selection bias • Regression to the mean effects

  43. Spurious associations can occur because of: • Seasonal staffing fluctuations • Seasonal fluctuations in illness • Examples: • Initiating a MRSA control program as the influenza season was ending • Fluctuating effectiveness of influenza vaccine programs due to year-to-year differences in virulence of virus and vaccine-virus match • Solutions • Exam effects of interventions across seasons (years) Emerg Infect Dis 2006; 12(6): 894-9

  44. Internal Validity Problems • Confounding factors, history • Trends/maturation effects • Seasonal effects • Detection bias • Attrition bias • Selection bias • Regression to the mean effects

  45. Ventilator-Associated Pneumonia Rates: UW Hospital (Hypothetical) ICP #1 ICP #2 ICP #1 VAP Rate (per 1,000 vent-days) UCL Date

More Related