"Improving Improvement: Building and Sharing Best Evidence for Clinical Decision-making"

1. "Improving Improvement: Building and Sharing Best Evidence for Clinical Decision-making" Paul Batalden, M.D. Stephen Liu, M.D. Dominic Geffken, M.D. Travis Harker, M.D. (Frank Davidoff, M.D.) Dartmouth-Hitchcock Leadership Preventive Medicine Residency Program Center for the Evaluative Clinical Sciences Dartmouth Medical School February, 2006

2. Frank Davidoff David Stevens Don Berwick Gene Nelson Gerry O’Connor Mark Splaine Stephen Plume Margie Godfrey Wayne Dysinger Carolyn Murray Marvin Dunn Tina Foster Greg Ogrinc David Aaron Dan Eubank Tom Colacchio Paul Gardent George Anderson Jeff Davis Linda Kinsinger Bob Harmon David Leach & Many others… Acknowledgements

3. Session Objectives: • Describe the shortcomings of current publications of quality improvement work. • List the key elements and criteria necessary for the publication of quality improvement work. • Identify how use of the elements and criteria described in this session will help improve your own improvement work and associated publications. • Illustrate how these understandings can integrate with a Preventive Medicine residency program.

4. Aim • To preview and illustrate the knowledge systems involved in improving patient care…and their implication for publication and for residency education.

5. Assumptions • We know that clinical care could be better than it is—and it’s frustrating to keep getting that news. • We all want to improve clinical care. • It is hard enough to improve personally, much less try to teach others how to change the world or the setting we work in—better to take up digital photography, woodworking or knitting. • We are able and willing to suspend what we usually defend for a few minutes to explore some slightly different ways of thinking, if it’s interesting.

6. The way we use scientific evidence in the care of individual patients.

7. When we want to use evidence to improve a patient’s situation… Generalizable Scientific Knowledge Particular Patient Measurable Improvement +

8. The way we can use that same thinking about improving the care for a population of patients.

9. When we want to use evidence to improve the usual care for a population of patients… Generalizable Scientific Evidence Measurable Performance Improvement Particular Context +

10. Process? System failures? Habits? Traditions? Population? Professionals? Setting? Successful changes? Hot buttons? Frustrations? Social supports? Building knowledge of the particular context, setting, population

11. All patients admitted to DHMC with a primary diagnosis of pneumonia from July 1 to September 30, 2003 59 patients identified 11 excluded 6 charts not available 2 pediatric cases 3 no initial diagnosis of CAP Population – Who are our CAP patients?

12. Patient Age: Average: 70 years (range 26-100 years) Length of Stay: Average length of stay: 8 days (range 1-45 days) Population – Who are our CAP patients?

13. Population – Where are CAP patients coming from? Emergency Department 63% Outpatient Clinics 15% Transfers From Outside Hospitals 22% DHMC Inpatient Ward

14. General internal medicine (GIM) 51% Hematology/Oncology 17% Cardiology 10% Pulmonary 10% Family practice 7% Nephrology 3% Rheumatology 2% Total 100% ICU/CCU admissions:10.4% Professionals – Who takes care of CAP patients?

15. Patterns – How do we take care of our CAP patients? Emergency Department 3.5 hours – 67% within 4 hours GIM Clinic 9.5 hours – 0% Transfers From Outside Hospitals 10.4 hours – 0% Average time from patient arrival to first antibiotic therapy Percentage of patients receiving antibiotics within 4 hours

16. An Actual Patient Admission • 2:00 PM The patient arrives for her scheduled appointment. The provider performs a physical exam and orders lab tests and a chest x-ray. • 2:45 PM The patient reports to the outpatient lab • 3 - 4:00 PM The patient goes to the outpatient radiology department where she receives a chest x-ray • 4:00 PM The provider reviews the labs and the CXR. A diagnosis of pneumonia is made • 5:00 PM Admitting, the admitting attending and the resident team are called and notified of the admission • 6:30 PM The patient arrives to her inpatient room on One East • 7:05 PM The admitting team writes admission orders for the patient • 10:15 PM Azithromycin 500 mg IV is administered • 12:00 AM Ceftriaxone 1 gram IV is administered

17. Habits? Time had not been an important consideration in treatment for CAP Traditions? Defer treatment decisions to the admitting residents for “teaching” purposes Inpatient treatments were not usually given in the outpatient clinic Building knowledge of the CAP context, setting, population

18. Frustrations? Significant delays in admitting patients from the ED and clinic were occurring due to a lack of inpatient bed openings Loss of information and lapses in treatment were occurring during patient “handoffs” and transfers of care Multiple admissions to an on-call team created delays in the admission process Building knowledge of the CAP context, setting, population

19. System Failures? Antibiotics ordered as QD were given the morning after admission Not ordered as first dose now Antibiotic change from a q8 to q24 medication resulted in inappropriate administration Multiple doses of q24 antibiotics – in ED and on the floor within several hours Building knowledge of the CAP context, setting, population

20. A clear question? About the evidence: Control for context? Design tests for generalizability? Careful search for biases? Statistical analysis to guide inferences? Peer review prior to publication? Stored in accessible format(s)? Building generalizable scientific knowledge

21. Guidelines exist on “Ideal” Treatment for CAP (IDSA, AHRQ, ATS, ACEP) Diagnostic Tests (Blood Cultures, Sputum GS) Appropriate Antibiotics Timely Antibiotic Treatment Prevention Vaccinations Smoking Cessation Advice What is ideal care for CAP patients?

22. Timing of antibiotics matters Patients who received antibiotics within 4 hours had an adjusted OR for mortality of 0.83 (0.72-0.96) The mean length of stay was also 0.4 days shorter Houck PM, Bratzler DW. Timing of antibiotic administration and outcomes for Medicare patients hospitalized with pneumonia. Archives of Internal Medicine 2004; 164:637-644. Building generalizable scientific knowledge about CAP

23. Choice of antibiotics matters Patients who receive an antibiotic regimen consistent with IDSA guidelines had a 26-36% reduction in 30 day mortality rates Gleason, PP, Meehan TP. Associations between initial antibiotic therapy and medical outcomes for hospitalized elderly patients with pneumonia. Arch Int Med. 1999; 159:2562-2572. Building generalizable scientific knowledge about CAP

24. Vaccinations are effective Influenza vaccination has an efficacy of 53% for preventing pneumonia, 50% for preventing hospitalization, and 68% for preventing death Gross PA, Hermogenes AW. The efficacy of the influenza vaccine in elderly patients: a meta-analysis and review of the literature. Ann Int Med 1995; 123:518-27 Pneumococcal vaccination is estimated to have a 47% efficacy for the prevention of pneumococcal bacteremia Shapiro ED, Berg AT, Austrian R, et al. The protective efficacy of polyvalent pneumococcal polysaccharide vaccine. N Engl J Med 1991; 325: 1453–60. Building generalizable scientific knowledge about CAP

25. Variation over time? Graphical displays? Balanced measures of outcome? Concurrent, real-time measurement? Measurement error control—operational definitions? Measurement design fidelity? Measuring performance improvements

26. Percentage of patients receiving antibiotics within four hours of arrival to DHMC Percentage of patients receiving appropriate antibiotics Percentage of patients receiving blood cultures prior to antibiotics Process and Outcome Measures of Quality for CAP

27. Percentage of patients receiving oxygenation assessment on admission Percentage of eligible patients receiving a pneumococcal and influenza vaccination Percentage of eligible patients receiving smoking cessation advice or counseling Length of stay Mortality rate/Discharge disposition Process and Outcome Measures of Quality for CAP

28. Community-Acquired Pneumonia Performance Measures

29. Community Acquired Pneumonia Outcome Measures

35. Standardization? Prompting strategies? Algorithms? Academic detailing? Resequencing work steps? Relationship work? Conflict management / negotiation strategies? Automation? Eliminate steps? Combine/couple steps? Change process ownership? Other? Getting the plan right for connecting generalizable scientific knowledge to particular context • Clarify aim(s) • Identify alternatives: 3. Select the best fit

36. Getting the plan right for connecting generalizable scientific knowledge to particular context Aim • Get correct treatment reliably started in less than four hours after arrival. • Prevent unnecessary CAP by improving immunization and smoking cessation care.

37. Start treatment where the patient is when diagnosis is made Develop standard admission orders and antibiotic guidelines Create documents that facilitate communication between outpatient and inpatient providers Include prompts for necessary vaccinations and smoking cessation in the admission orders Provide individualized feedback to providers about the care that they provide to pneumonia patients Alternatives for improving the care for CAP patients

38. Local and non-local strategies of importance? “Ways” things get done? What matters to people who work here, how, why? Leadership of change history? Ways leaders manifest their support? Sustaining / embedding strategies? “Unlearning” needed? Supports for same? Relation of measurement to operations? Change ownership? Clinical policy ownership? Log of change process? “Making it happen”

39. “Ways” things get done? Needed input and support from ED, GIM Clinic, One East, ID, Pharmacy, Nursing, and Residents Needed approval of various committees for standard order forms Needed support of administration National and local priority Knowledge of “how to make it happen” at DHMC

40. What matters to people who work here, how, why? Infectious Disease very concerned about possible misuse of standard orders have been frustrated in the past by the lack of diagnostic testing are primarily concerned with relatively rare causes of pneumonia Knowledge of “how to make it happen” at DHMC

41. What matters to people who work here, how, why? Emergency Department Looking for ways to determine the appropriate antibiotics for CAP patients Have not traditionally used written orders Knowledge of “how to make it happen” at DHMC

42. What matters to people who work here, how, why? General Internal Medicine Concerned about keeping up with busy schedules while treating patients who need to be admitted Increasing problem of patients waiting in the clinic for an inpatient bed Knowledge of “how to make it happen” at DHMC

43. Was there any measurable improvement?

44. Community-Acquired Pneumonia Performance Measures

45. Community-Acquired Pneumonia Performance Measures

46. Community Acquired Pneumonia Outcome Measures

47. Knowledge systems involved • right plan • IV • making it happen • V Generalizable Scientific Evidence Measurable Performance Improvement Particular Context + • control context out • design for generalizability • I • focus on particular / identity • II • time • measurement • fidelity • III

48. Brief break

49. Toward stronger evidence on quality improvement How can publication guidelines help? Davidoff, Batalden

50. Quality Improvement: A three component engine • Scientific knowledge • Competent individual and system performance • Knowledge about how to acquire and increase competence Davidoff, Batalden