Exploring the Relationship Between Managerial Practices and Quality Improvement

1. Exploring the Relationship Between Managerial Practices and Quality Improvement William Berry, MDPrinciple Research Scientist, Harvard T.H. Chan School of Public HealthDeputy DirectorAriadne Labs

2. The South Carolina Ariadne Labs Partnership • 4+ years to improve OR Safety • The work has improved patient care across the state • Some observations • SC hospitals are pioneers • SC hospitals are different from one another • Some hospitals appear more ready to work quality improvement work than others • And…..

3. Management Matters

4. The World Management Survey: A Little History • Developed by Stanford, London School of Economics, and the Harvard Business School • More than 15,000 interviews representing 34 countries • Across multiple industries: • Manufacturing • Schools • Healthcare

5. Asked if organizations adhere to 3 best practices: • Targets: Does the organization support long-term goals with tough, but achievable short-term performance benchmarks? • People Management: Does the organization properly manage and retain talent while re-training or moving underperformers? • Monitoring: Does the organization rigorously collect and analyze performance data to identify opportunities for improvement?

6. Improving Management Practices In Textile Manufacturers in India • Randomized study of Indian textile manufacturing firms • 28 plants across 17 firms randomized to intervention (diagnostic survey plus four months targeted consulting) or control (survey alone) in 2:1 ratio • 17% increase in productivity with ~$300,000 increased productivity at intervention plants

7. l Manufacturing Was Found To Have Better Management Practices Than Hospitals United States

8. The Management Survey in Healthcare • 2,000 Acute Care Hospitals in 9 Countries, including the U.S. • Researchers cold called hospitals • One interview per hospital, targeting a middle manager (typically nurse managers) • Interviews focused on the adoption of management practices: • Monitoring • Targets • People Management

9. The Findings In Healthcare • Management correlated with hospital quality of care, productivity, and financial performance • Management practices varied widely across hospitals

10. Management is correlated with quality of care: Higher management scores associated with lower AMI mortality rates (risk adjusted) Notes: Based on 324 observations with available AMI information (Canada:29; Sweden: 48; UK: 74; US: 178). We z-score the AMI data within country to take into account differences in the way the AMI rates are calculated across countries, and keep only hospitals with at least 20 AMI cases in a year. For both AMI rates and Management, we take residuals from a regression including country dummies, hospital controls(number of employees, specialty, percentage of managers with a clinical degree), noise controls (13 interviewer dummies, the seniority and tenure of the manager who responded, the duration of the interview, and an indicator of the reliability of the information as coded by the interviewer, interviewee type) and regional dummies. AMI mortality rates data refer to 2009 in the US and UK, to 2008 in Sweden and the average between 2007 and 2009 in Canada. The p-value on the difference between the bottom and the middle tercile is 0.204; the p-value on the difference between the bottom and the top tercile is 0.001. The p-value on the difference between the middle and the top tercile is 0.07.

11. The Management Survey in South Carolina

12. South Carolina Management Survey • Project launched in 2014 • 15 participating hospitals to date • 84+ of interviews conducted to date • Recruiting hospitals that perform inpatient surgery What is different about the work in SC? • Looking at organizations more deeply than ever done before • Multiple interviews and across service lines: • C-suite, Chiefs of Departments, Middle Managers • Assessing additional management practices than previous projects • We can explore the relationship that management has to clinical outcomes

13. Managerial Practices Assessed In The Interviews • 4 practices: • Operations: whether standardized protocols are understood and used by all clinical staff • Monitoring: whether performance is tracked and reviewed and whether differing levels of performance lead to different consequences • Targets: whether established goals address long- and short-term time horizons and are clearly defined and tied to hospital objectives • People: whether high and poor performers are adequately rewarded and penalized and talent is properly managed and retained • Key Performance Indicators • Management believed to matter on multiple levels • Financial and clinical outcomes • Process improvements (e.g. uptake of surgical safety program across hospitals in Safe Surgery 2015 Collaborative)

14. Massachusetts is following in the footsteps of South Carolina

15. Some preliminary findings from South Carolina and Massachusetts

16. C-suite managers believe that the management practices are better than middle managers C-Suite Middle Managers

17. C-suite scores are not aligned with middle managers’ scores Line of agreement Note that in only one hospital are middle managers more optimistic than C-suite managers.

18. Better key performance indicator tracking by C-suite is associated with better management scores C-Suite Middle Managers Key Performance Indicator (KPI) Use Note: Based on a sample of 147 interviews across 26 hospitals in South Carolina and Massachusetts. Higher values = better use of KPIs

19. We want to expand this work to more hospitals in South Carolina

20. Benefits of participation • Individualized, benchmarked report for each hospital that participates • Understand management variation within your hospital • Benchmark management practices in your hospital to others in your state and beyond • Identify areas for improvement • Help build knowledge around management and clinical quality improvement

21. Sample Report

22. Overall Hospital Management Score • The overall management score is calculated using using all interviews conducted within each hospital. • A management score of 5 = full adoption of processes, 0 = No adoption of processes

23. Alignment Across Management Areas Four Management Areas: Operations management – whether standardized protocols are understood and used by all clinical staff Monitoring management – whether performance is tracked and reviewed and whether differing levels of performance lead to different consequences Target management – whether established goals address long- and short-term time horizons and are clearly defined and tied to hospital objectives. People management – whether high and poor performers are adequately rewarded and penalized and talent is properly managed and retained. A management score of 5 = full adoption of processes, 0 = No adoption of processes

24. Joining The Project Your Organizations Commitment: • 7-10 one-hour interviews with managers across your organization • Designate a contact for your organization to work with our team to schedule the interviews How To Join The Project: • Information about this project is included in your meeting materials • Talk to Alex Haynes or myself at this meeting • Contact our team via email for more information: • Alex Haynes, MD abhaynes@ariadnelabs.org • Brooke Huskeybhuskey@ariadnelabs.org

25. Thank you

26. Extra Slides In Case You Need Them

27. l Hospital distributions typically shifted the the left relative to manufacturing US Italy India

28. C-suite managers are more optimistic about their organization C-suite managers tend to report higher rate of adoption of basic management practices relative to middle managers C-suite managers scores are much more compressed relative to middle managers. In particular, C-suite managers tend to have a much thinner left tail in the distribution (and no scores below 2).