Informing for Improvement Report cards, performance measures and quality indicators – why bother?

1. Richard Hamblin Center for Health Studies Group Health Cooperative of Puget Sound Informing for ImprovementReport cards, performance measures and quality indicators – why bother?

2. Why publish report cards? • How do we encourage consumerism? Well, one thing you do is you make sure people understand their options… • G.W Bush, Minneapolis, August 22, 2006 • Our choice information pages …help you make a choice that best suits you.” • UK Department of Health choice website

3. But… • “when this information is published only a minority are aware of it; of those, most do not understand it, trust it or use it…” • Gwyn Bevan 2005

4. My objectives • Understand this contradiction • Group health provides a unique population • Chronic conditions • My Group Health • What is the likely use of such information? • What does this imply about presentation and dissemination? } More enthusiastic users?

5. How might it work • Pressure on providers • Patients as consumers – choosing the best providers • Informed and empowered patients – getting the best from their providers • Which of these two is most likely to work?

6. Survey • Written survey tool mailed to 600 respondents My GH user n = 300 My GH non-user n = 300 My GH user diabetes n = 150 My GH non-user diabetes n = 150 Diabetes n = 300 My GH user no chronic condition n = 150 My GH non-user no chronic condition n = 150 no chronic condition n = 300

7. Survey • 22 written questions to test pre-determined hypotheses • Patients with long-term conditions are more interested • My Group Health users are more interested • Less satisfied patients are more interested • Information more likely to be used for boosting confidence than changing provider • Data collection February to April 2007

8. Responses • Ethnicity similar across groups • MyGH users wealthier and better educated • Diabetics generally lower income

9. Lots of interest • 11 point scale (0-10) used to report interest

10. But little prior knowledge or use No meaningful differences between the groups

11. What do I do with this then?the continuum of potential use Active and immediate None “Change doctor inside current plan” Active Consumerist Passive/ uncertain None Prospective Active informed patient “Understand better how my doctor rates” “Choose doctor when entering health plan” “I would not use” “Boost confidence to discuss things I don’t understand or agree with.”

12. Proportion of respondents citing different uses for data (forced choice of one use) For all groups “understand better” is a significantly greater proportion than any other Use versus self-reported interest does not vary (except for the would not use group)

13. n Boost confidence Change doctor MyGH* 21 8 Diabetes* 22 9 Active users only • Surprising result – expected a greater proportion in the active section • Active uses only – significant results (p<0.05) • “Boost confidence” a more common response, but very small numbers

14. Key result • Just because people are interested in the information doesn't mean that they are going to use it to make choices tomorrow

15. % of responders Individual measures Individual physicians Benchmarks Total 45.9% 57.8% 76.1%* How do patients want data presented? • Individual measures vs an overall service rating • Individual physicians vs hospital/practice • Benchmarks/expected performance vs rankings • * (p<0.01)

16. r2 Total 0.000 MyGH 0.002 NMyGH 0.011 Diabetes 0.001 No CC 0.001 How does satisfaction affect interest? • Are satisfied patients less interested in having information about quality? • Test 1: Correlation of interest scale with CAHPS satisfaction scale Correlation between interest and satisfaction ratings

17. Always Not always Explains 7.7 7.7 Listens 7.6 7.7 Respects 7.7 7.5 Time 7.5 7.9 How does satisfaction affect interest? • Test 2: Comparison of interest scale with specific CAHPS attributes of patient-focused care Mean interest scores by regularity of CAHPS attributes

18. CONCLUSIONS • Many prior expectations were wrong • Interest in performance information uniformly high • Prior knowledge and use of report cards uniformly low • Contrary to expectations, little difference between groups • No relationship between satisfaction with doctor and desire for information about quality

19. CONCLUSIONS • The importance of “better understand” • Doesn't have to be “used” to be “useful” • Possible interpretations • Information as a resource • Reassurance • Accountability • Understanding as a precursor of action

20. CONCLUSIONS • Large majority in all groups favoured comparisons with benchmarks rather than ranked performance • Consistent with the “use” finding

21. Policy implications – consumerisms’ weaknesses and an alternative approach • Publishing information about quality will not necessarily encourage choice • Not because the data are badly presented but because most patients don’t prioritise choice • A different goal of trust and understanding of quality of service

22. Policy implications – unresolved next steps • Balancing measurement of different things: • Clinical process • outcome • experience • How to set external benchmarks? (e.g. NQF process) • How to determine the “normal range”? (e.g. outliers, composites)