Performance management in the NHS: going beyond the metrics

1. NHS Confederation / LSE Health seminar, 27 April 2010 Performance management in the NHS: going beyond the metrics Dr. Mark Exworthy Royal Holloway-University of London M.Exworthy@rhul.ac.uk

2. Performance: opening comments • Performance has been a dominant narrative within English health policy reform in the last decade • Focus is shifting over time* • Performance management shortcomings* • Research tends to… • Focus on technical `performance products’ • Lack conceptual development • Wider perspective on performance required… • To go beyond the metrics

3. Performance management: a shifting focus over time Exworthy et al, 2010

4. Performance: opening comments • Performance has been a dominant narrative within English health policy reform in the last decade • Focus is shifting over time* • Performance management shortcomings* • Research tends to… • Focus on `performance products’ • Lack conceptual foundations • Wider perspective on performance required…

5. Performance management shortcomingsAdapted from Sheaff et al, 2004; Talbot, 2005 • Incompleteness • Over-complexity • High transaction costs • Attribution difficulties • Quantity-quality imbalance • Gaming • Short-term focus • Performance churn

6. Performance: opening comments • Performance has been a dominant narrative within English health policy reform in the last decade • Focus is shifting over time* • Performance management shortcomings* • Research tends to… • Focus on `performance products’ • Lack conceptual foundations • Wider perspective on performance required…

7. Wider perspectives on performance • The influence of subjective views of performance upon management • Conceptual perspectives on performance • Consequence (intended and otherwise) of performance management

8. 1. Formal & informal performance Formal performance Hard information ~ official metrics Quantitative measures; retrospective Tends to focus on `poor’ performance rather than improving good performance Example: rankings, league tables, targets `Safety net’ function Informal performance Soft information ~ perceptions, founded on subjective judgements Qualitative measures, can be prospective; reputation, trust, goodwill, tacit knowledge, credibility Eg. `safe pair of hands’, `keep an eye on them’, `what is really happening?’ Substitution & complementary functions

9. Formal performance Safety net: formal notions of performance invariably used as a `safety net’ to address organizations with poor (formal) performance Incentive? Formal performance may offer `high performing’ organisations little incentive to improve High performance did not ensure freedom from centre Informal performance Substitute: informal performance is often deemed more responsive, timely and useful than formal performance Complement: both formal and informal performance were seen as important in assessing organisations Adapted from Goddard, Mannion & Smith, 1999; Exworthy et al, 2010a • Formal performance is insufficient to explain existing performance patterns and to promote improvement • Need to understanding better the interplay between formal and informal performance

10. Example #1. FTs exercising their autonomy • FTs have not `performed’ as well as expected • Is autonomy such a panacea after all? • FTs have technical ability to exercise autonomy • Many FTs lacked the willingness to do so: • Some de facto autonomy already exists • Greater risk • Uncertain rules of the new game • Legitimacy • Fear of negative impact on local health economy • Need to explore FT managers’ motivations, attitudes towards the award and use of autonomy Exworthy et al, 2010a

11. Example #2. Mid-Staffordshire NHS Foundation Trust • An FT and a high performing organisation? • “In the four years from 2002 until 2005 (the last year of the star ratings system which ranked trusts from 0- to 3-star), Stafford had got, respectively, 2, 3, 0 and 1 star. Yet it was encouraged or "invited" to seek FT status ” (Paton, 2010http://www.publicservice.co.uk/feature_story.asp?id=13870) • Patients and staff knew about `poor’ performance: • “I remember at the time when our staffing levels were cut and we were just literally running around. Our ward was known as Beirut from several other wards. I heard it nicknamed that. ITU used to call us Beirut… I remember saying: this will have repercussions, this can’t go on like this. Because relatives were regularly coming up to us and saying: my Mum has been buzzing for this long, there has been a buzzer going there for that long.” (p.197) Francis Inquiry report, 24 February 2010 http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/@dh/@en/@ps/documents/digitalasset/dh_113447.pdf

12. Example #3. Performance management in local health economies Director, PCT: “We’ve got a wealth of informal knowledge about [Hospital X] because there are a number of people in the team who’ve been there before and the nature of [Hospital X] is it’s been very open to it because they’ve been so desperate for help. [Hospital Y] is much more of a closed book to us. There are people that have come into different roles in the PCT who may have known something about it but it always feels quite antagonistic that relationship and the GPs are pretty disenfranchised lot as well. So there’s not a lot [of PCT staff] that has a kind of root of understanding and influencing this... isn’t there either. [Hospital Z] was a fairly sort of collegiate comfortable sort of relationship and there’s a fair amount of sort of, traffic, you know, informal networking stuff through that, for a lot of people in the PCT or- there are in the Exec Team of the legacy PCTs” http://www.sdo.nihr.ac.uk/projdetails.php?ref=08-1618-125 p.153 Exworthy et al, 2010

13. 2. Disclosure of clinical performance • “The more closely we are watched, the better we behave?’ (Jeremy Bentham) • “Sunlight is the best of disinfectant; electric light the most efficient policeman” (Brandeis)

16. Disclosure of clinical performance: unintended consequences • Quantification emphasis • Short-term objectives dominate • Manipulation of performance data and behaviour adjustment • “Misleading inferences” could be drawn from “raw performance data.” • Organisational inertia Smith, 1995

17. Public disclosure of performance data Problem identification Naming of individual Public sanction Recipient response Pawson et al, 2005, S1:23

18. Public disclosure of performance data(with unintended consequences) Problem identification Naming of individual Public sanction Recipient response Culprit misidentification Dissemination dissimulation Sanction misapplication Unintended outcome Pawson et al, 2005, S1:23

19. Public disclosure of performance data i. Identification • Initial focus on mortality rates of cardiac surgery • Disclosure supported by profession • 2-3% mortality rate neglects most patients • Attribution issue with 30 day mortality rate • Limited use of comparisons ii. Naming • Average age of cardiac surgery patient = 68 years (and rising) • Data accessibility and user literacy • Named consultant `hides’ clinical team

20. Public disclosure of performance data iii. Public sanction • Strong normative pressure for clinicians to participate but not compulsory • 25% cardiac surgeons do not participate • Little evidence of `choice’ as sanction • Sanction mediated by user proxy (GP) • Episodic nature of care & emotional concerns iv. Recipient response • Initially educational but increasingly judgemental • Professional ownership and promotion • Surgeons’ sense of autonomy threatened • Danger of gaming but little evidence so far • Risk aversion: high risk patients avoided? • Some rejection of performance measures • Junior surgeons less exposed to high risk cases

21. Disclosure and performance management • Patients’ experiences: • PROMs • Financial incentive: “Hospital income will increasingly be linked to patient satisfaction, rising to 10 per cent of their payments over time” (DH, NHS 2010-2015: From good to great. 10 December 2009) • Danger that performance measures becomes too esoteric • Management remains loosely coupled with clinical performance • Patients unable to interpret performance data

22. A final thought on performance “There is a need to recognise the imperfections and limitations of [performance] measures, and to use them as a means of supporting politically informed judgements” Stewart and Walsh, 1994, p.45