Benchmarking and Performance Indicators

1. Benchmarking and Performance Indicators Overview Jay Ford, PhD

2. Agenda • Framework for benchmarking • Role of benchmarking • Example benchmark measures • Importance of comparative feedback • Process Steps • Review of a current benchmarking initiative

3. Some is not a number, soon is not a time. -- Don Berwick, MD

4. Key Definitions • Gift and Mosel (1994) define benchmarking as “the continual and collaborative discipline of measuring and comparing the results of key work processes” • A performance or key performance indicator (KPI) is a measure of performance. • Performance measurement is the process used by an organization to establish the parameters to measure if programs are reaching the desired results

5. Who uses data? • Decision Makers • State (e.g., SSA or Legislature) • Funding Agency(e.g., State or Managed Care) • Provider (e.g., Executive Director, Board) • Consumers • Clients • Employers • Employees

6. Data is important to the …. • … state because they need to show that treatment is working • … legislative body because they want to be sure that citizens are receiving a quality services for their investment • … managed care companies who are interested in quality care

7. Data is important to the …. • … consumers who need to get into treatment now and want to receive quality care • … employees who need to understand that their efforts are beneficial to the agency and clients • … employers seeking quality care for their employees

8. Data is important to the …. • … executive director who needs to effectively and efficiently run the treatment agencies

9. Alignment Outside Influences Inside Influences Consumer Accreditation NOMS Board Payer (State or Managed Care Employers Management Employees

10. The challenge is to bring it all together to create a cohesive picture within your organization

11. Performance is measured in all settings How helpful is performance data? Does data drive organizational vision? Does data drive organizational decision-making? The Measurement of Organizational Performance

12. The Limitations of Outcomes • A thermometer reading would be of no value as a measure of your health… • If you didn't know that 98.6 is the “average” temperature!

13. The Limitations of Outcomes in Your Organization • Similarly, knowing that your missed appointment rate is 27% is of little value in evaluating the health of your agency…if you don’t know how others are performing

14. Data without Meaning • We are deluged with data without meaning • Data without a context is just a bunch of numbers • What is the answer?

15. The Role of Benchmarking • Benchmarks provide the vital internal and external context for understanding your outcomes

16. Descriptive Benchmarking Comparative Benchmarking Process Benchmarking Each builds on the preceding type A Tiered Model of Benchmarking

17. Most common form of benchmarking Industry overviews, government reports Static snapshot of industry performance No formal or statistical comparisons between an organization’s performance and industry norms “Eyeball analysis” can be misleading Descriptive Benchmarking

18. Example Descriptive Benchmark

19. Example Descriptive Benchmark Report

20. Formal comparison of an organization’s performance against a descriptive benchmark Typically reported as percentile rankings but can be reported in graph form also Comparative Benchmarking

21. The Importance of Formal Comparative Benchmarks

22. Benchmarking differs from formalized research or laboratory science Benchmarking views data as a vehicle rather than a destination Benchmarking generates ideas and hypotheses---not answers Benchmarking is a communal activity Important Benchmarking Concepts

23. Human nature gives benchmarked data its power Raw data tells no stories Benchmarked data draws comparisons Nobody wants to “bring up the rear” Benchmarked data prompts action Benchmarking as an Impetus to Action

24. The Limitations of Benchmarking • Benchmarks profile your organization against others and help identify opportunities for improvement

25. Improve? How? • Benchmarks do not give any indication as to how to improve

26. Based on a simple premise: The methods of top performers differ from those of others Process benchmarking systematically compares tactics of top performers with those of others Methods that distinguish between top performers and others may be regarded as potential best practices The “How”: Process Benchmarking

27. Surprises Abound • What is expected usually does not materialize • Top performers do not know why they perform well • Subtle factors often prove to be powerful

28. All concepts also apply to internal benchmarking Descriptive, comparative or process benchmarking can be conducted within an organization Examples of internal comparative benchmarks: Productivity differences among staff Access differences among locations Unit cost differences among programs Staff satisfaction differences among leaders Internal Benchmarking

29. Looking at the Whole Picture Clinical Business

30. Types of Benchmark Measures Time & Attendance Scheduling Materials Mgmt Registration Billing Payroll

31. Example Benchmark Measures • Business • Days between 1st contact and assessment • Days between assessment and 1st treatment • Successful transition across levels of care • Clinical • Engagement or Retention • No-show or cancellation rates • Client • Satisfaction • Family engagement

32. Example Benchmark Measures(cont) • Financial • Cost per unit of service • Payer mix • Net days in account receivable • Days cash on hand • Productivity • Average caseload size • Average group size • Staff turnover

33. Identify potential best practices through process benchmarking and other vehicles Enhance organizational performance through NIATx principles Emphasis on shared learning from one another and from “top performers” Moving Beyond “the Numbers”

34. Comparative Feedback • Understand the whole picture • Select a few key outcome measures • Use of reports to guide questions • Benchmarks vs. Targets • Focus on the comparison (internal vs. external)

35. Comparative Feedback • Measurement Comparisons • Performance vs. Outcomes • Business Process vs. Treatment Performance/Outcomes Importance of comparisons • Types of Feedback Reports • Data Quality • Performance Reports • Pay for Performance

36. Comparative Feedback • Organizational Performance versus • a target (internal) or • a benchmark (external) • Types of comparisons • Internal comparisons over time • External performance comparisons to other similar organizations • External performance comparisons to other agencies within a state

37. Two Quick Examples • Dashboard Reports • Oklahoma Department of Mental Health and Substance Abuse Services

38. Benefits of a Dashboard • Set realistic objectives and create buy-in • Use of data maintains quality • Support from outside the organization • Help build referrals and funding • Requires human interpretation

42. Steps of Developing a Benchmarking System

43. SAAS, NIATx, and Behavioral Pathway Systems are partnering to sponsor an addiction-specific national benchmarking initiative BPS specializes in behavioral health and human services benchmarking and has numerous state and national benchmarking initiatives underway Benchmarking for Organizational Excellence in Addiction Treatment

44. Current Benchmark Domains • Operational Benchmarks • Clinical Benchmarks • Organizational Climate • Financial Benchmarks

45. Resources • Gift, R. and Mosel, D. (1994). Benchmarking in Health C are, American Hospital Association • Migas N (2010) Let dashboards lead you to success. Behavioral Healthcare. February 2010, 12-15 • Knopf, A (2010) Magellan, Maricopa pass contract midpoint. Behavioral Healthcare. February 2010, 16-19 • Lefkovitz PM, Ford J, Vaughn B and Nance R (2009). Promoting benchmarking in addiction treatment. Behavioral Healthcare. 29(4): 28-30.