Studying hospital quality using mixed methods

1. Studying hospital quality using mixed methods Elizabeth H. Bradley, PhD Yale University School of Medicine

2. Disclosure and Acknowledgments The research is funded by the Agency for Health Care Research and Quality (#R01HS10407-01), and the Donaghue Medical Research Foundation (#02-102) Conducted in collaboration with Genentech and the National Registry of Myocardial Infarction (NRMI) Investigators

3. Purpose of the presentation Provide an example of using mixed methods to identify determinants of hospital quality Highlight the benefits and special challenges in employing mixed methods

4. Background ACC/AHA guidelines recommend beta-blocker prescription for patients hospitalized with acute myocardial infarction (AMI) However, many patients do not receive beta-blockers after AMI, and hospitals vary substantially in rates of beta blocker use and improvement in those rates over time

5. Improvement in hospital beta-blocker rates, 1996-1999

6. Objective To identify success factors in hospitals’ increased rates of beta-blocker use for patients with AMI… what works?

7. Mixed methods Qualitative study Site visits in higher/lower performers In-depth interviews with key staff Quantitative study Closed-ended survey of random sample of hospitals

8. Qualitative study Site visits with in-depth interviews (n=45) in 8 hospitals with higher versus medium/lower performance in beta-blocker rates • 14 physicians • 15 nurses • 11 quality improvement staff • 5 administrators (senior and mid-level) Constant comparative method for data analysis of qualitative data

9. A taxonomy to classify QI efforts along key dimensions Goals – content, specificity, sharedness Admin support – philosophy, resources Clinical support – physician, nurse, ancillary Systems design – standing orders, pathways, reminders, care coordinators, etc. Data feedback – validity, timeliness Contextual factors – size, teaching status, system affil, financial constraints, market and regulatory pressures, etc.

10. Hypotheses about “what works” In beta-blocker performance, presence of clinical champions and administrative support for quality improvement are more important than systems design interventions Data feedback, especially when it is physician-specific, can be viewed as punitive and can backfire as an improvement effort

11. Quantitative study Cross-sectional study of 234 hospitals from those participating 30+ months during Apr 96-Sept 99 in the National Registry of Myocardial Infarction (reflects 54.2% response rate) Patients: 60,363 treated for AMI in these hospitals during 1998-1999, the years just after beta-blocker recommendations were widely published Telephone survey of QI directors at each hospital Hierarchical models to estimate p (high-rate hospital)

12. Hospital sample (n=234) Beta-blocker rates Mean, range 60%;19% - 89% Urban 83% Teaching 39% Annual AMI volume (median) 137 patients

13. QI efforts Type of QI effort Prevalence Standing orders 57% Clinical pathways 58% Educational efforts 76% QI teams 80% Care coordinators 50% Reminder forms 28% Computer support 34%

14. QI efforts (continued) Type of QI effort Prevalence Data feedback reports 97% Quarterly reports 81% Public display of data 34% Data reports last 6 months 39% Data are physician-specific 11% Has physician champion 92%

15. QI efforts and performance QI effort Adj OR p-value Standing orders 2.3 .066 Physician champion 10.5 .001 Admin support [1-5] 2.0 .009 Data feedback that is physician-specific 0.1 .001

16. Qualitative study benefits The qualitative study augmented conceptual background for quantitative study: • Taxonomy with which to characterize QI efforts, a multifaceted intervention • Hypotheses about systems interventions, clinical champions, administrative support, and data feedback 3. Comprehensible language for survey design

17. Special challenges of using mixed methods Integration the qualitative and quantitative work – benefit comes from their integration but easy to split them off The “juicy” ideas in qualitative work can be difficult to measure (organizational change, culture, etc.) and test in larger samples Qualitative work often slows down and delays the quantitative work Publishing mixed methods – a special challenge (length, reviewers’ tolerance for unknown methods)

18. Why use mixed methods? “Grounds” our work, so that we ask the important questions, have realistic hypotheses, and use sensible language Increases the potential that research will be more easily translated into practice

19. Conclusions Mixed methods studies are particularly advantageous for some, but certainly not all, topic areas Research inquiries that involve multifaceted interventions, interdisciplinary interactions, or innovation and organizational change are good candidates for mixed methods