Reporting performance in England – Q or A?

1. Reporting performance in England – Q or A? Richard Hamblin Director of Intelligence Care Quality Commission Alberta Health Service 7 April 2010

2. Agenda • Contexts • The UK, its demography, politics and health system • How is reporting performance supposed to work? • Tin Openers and Dials • Success and Problems • Moving forward – tin openers revisited?

3. UK 4 countries • England c.50m • Scotland c.5m • Wales c.3m • N Ireland c.2m

4. UK - demography Aging population C 90% white, though large immigrant populations from Africa, Caribbean, Indian Subcontinent and East Asia esp. in London and major urban centres

5. UK – Economy 6th largest GDP 13th greatest GDP/capita Greatest income inequality in EU (2001) Economic structure (2007)

6. UK -Politics • Parliamentary democracy • Labour Government since 1997 • Devolution since 1998 – 4 countries/ 4 different health policies • Can this hold?

7. UK - National Health Service (NHS) Founded 1948 Free at point of care All services except: • Treatments of “unproven effectiveness” (NICE etc) • Non-NHS long-term/nursing home care • Co-payments • Prescription charges (c. $12 per item), dental, optician, charges • Various exemptions from payment Private insurance largely via employers, and almost entirely for rapid outpatient consultation and elective surgery (about 10% of population)

8. UK - Health spending US$2992 per head on health annually 8.4% of GDP in 2007 • Historically lower than EU average • Recent unprecedented growth (but only from 85% to 90% of OECD average) 82% government spending, 12% personal, 6% private insurance Government spending funded from general taxation (& National Insurance) OECD: 2007 data

9. Primary Care in the NHS The Quality and Outcomes Framework Effects of the Reforms The NHS The Collings Report Developments 1948 – 1997, New Labour NHS Spending “... health spending will increase by about 5% annually for five years...” “You’ve stolen my bloody budget!” Paying Physicians for Quality NPCRDC

11. The macro politics of health in UK The “four way bet” • England - deterrence orientated regulation and markets (with side bets on centralised IM&T development and collaborative quality improvement) • Scotland - professionalism and collaborative quality improvement within single system working • Wales - localism in terms of public health focus integrated with local authorities • N Ireland - history of politician-free managerialism • Nuffield Trust (2009) study suggests England won Greer 2004

12. The macro politics of health in UK The “four way bet” • England - deterrence orientated regulation and markets (with side bets on centralised IM&T development and collaborative quality improvement) • Scotland - professionalism and collaborative quality improvement within single system working • Wales - localism in terms of public health focus integrated with local authorities • N Ireland - history of politician-free managerialism • Nuffield Trust (2009) study suggests England won Greer 2004

13. The macro politics of health in UK The “four way bet” • England - deterrence orientated regulation and markets (with side bets on centralised IM&T development and collaborative quality improvement) • Scotland - professionalism and collaborative quality improvement within single system working • Wales - localism in terms of public health focus integrated with local authorities • N Ireland - history of politician-free managerialism • Nuffield Trust (2009) study suggests England won Greer 2004

14. The macro politics of health in England since 1997 • Pious hope - 1997 • Clinical governance - 1999 • More money and clear accountability (the NHS plan) - 2001 • Licensing and competition (pious hope redux?) - 2005

15. The macro politics of health in England in 2010 • Clinically led improvement (Darzi - 2008) plus • Public Information plus • Genuine regulation plus • Some degree of plurality and choice plus • Integration of health and social care

16. The role of the Care Quality Commission • The Care Quality Commission is the independent regulator of health and social care in England • We exist to make sure people get better care • We do this through: • Registering all providers of health and social care against legally enforceable standards and monitoring and enforcing to make sure that these standards remain in place • Regular reviews of performance • Special reviews of specific services • Publication of information

17. Tin openers and dials Concept from Carter and Klein (e.g. 1992) Tin openers open up cans of worms Dials measure things Most of the time you need to ask questions as much as you need to answer them

18. CQC using tin openers and dials Special Reviews Publishing Information

19. Regular reviews of performance • Single measures of single issues for individual providers • Basically are you meeting a politically set and mandated target

20. Successes and problems

21. This approach has had some success • Access • Hospital Acquired Infection • A natural experiment

22. Dramatic reductions in numbers waiting a long time for elective surgery

23. Emergency ambulance response times improve Ambulances responding to emergencies in 8 minutes

24. And MRSA too Baseline year Target year

25. Devolution and a natural experiment % patients waiting for hospital admission > 12 months Source: http://www.statistics.gov.uk National Health Service hospital waiting lists by region: Regional Trends 35, 36, 37 & 38

26. Problems • So given that success – why the worry • Gaming • Complexity and tripping over each other • “The synecdoche problem”

27. Gaming: ‘corrected’ response times ‘Corrections’’ only 2% to 6% 75% < 8 minutes 75% < 8 minutes Source: http://www.chi.nhs.uk/eng/cgr/ambulance/index.shtml

28. And yet… • The response in Wales was to set progressively less taxing targets • Scotland and Northern Ireland did nothing until 2007 • So saved lives and encouraged dishonesty?

29. The data issue • What happens when one piece of information becomes the be-all and end-all and you know what will be counted as good and bad? • “Any observed statistical regularity will tend to collapse once pressure is placed on it for control purposes” Goodheart 1984 • So beware of Hospital Standardised Mortality Rates

31. Confusion! Overlap conflict and confusion 13 New (ish) Information rich schemes Quality Account Service Line Reporting Registration Vital Signs World class commissioning Periodic review Comprehensive Area Assessment Payment by results CQUIN NHS choices Quality Framework/metrics NICE standards PROMS 5 Policy Drivers to improve quality Performance Management Commissioning Regulation Financial and Quasi-market Clinical leadership

32. A part is not a whole… • How realistic to describe a C$1bn organisation in one word? • You can’t measure everything • And some of what matters is hard to measure • So you end up with a small group of measures trying to describe everything • Distraction, management to measure • Value? • What happens when there are conflicting claims (e.g. CQC and Dr Foster November 2009)

33. An attempt at a solution – registration and genuine regulation

34. Objective - at all points of care People can expect services to meet essential standards of quality, protect their safety and respect their dignity and rights. Independent healthcare Single system of registration Single set of standards Strengthened and extended enforcement powers Adult social care 1 Registration 2 NHS 3

35. Registration timeline(subject to legislation) NHS Trusts April 2010 Adult social care and independent healthcare providers (CSA) Oct 2010 Primary dental care (dental practices) and independent ambulance services April 2011 Primary medical services (GP practices and out of hours) April 2012

36. The difference registration will make • All health and adult social care providers are meeting a single set of essential standards of quality and safety • Standards are focused on what is needed to make sure people who use services have a positive experience- a direct result of what people said they wanted • A single regulatory framework across health and adult social care; people receive safe and quality care no matter which part of the care system they experience and where

37. Benefits of registration • Outcomes - More outcome-based registration that protects and promote equality, diversity and human rights and makes providers accountable • Information -Improved access to timely, relevant and reliable information enabling consistent comparisons and promotion of joined up care • Enforcement - Earlier identification and swifter action to follow up concerns including enforcement action where necessary • Burden - Reduced unnecessary regulatory burden and associated costs of demonstrating compliance • Compliance - Increased compliance by health and adult social care providers • Process -Improved transparency, speed, consistency and reliability of registration

38. CQC’s guidance about compliance documents

39. CQC’s guidance about compliance • The regulations mapped to six outcome headings: • Involvement and information • Personalised care, treatment and support • Safeguarding and safety • Suitability of staffing • Quality and management • Suitability of management Our focus: Plain English People focussed Outcome Based

40. CQC’s guidance about compliance: example of an OUTCOME Safeguarding people who use services from abuse Plain English • OUTCOME 7 • What should people who use services experience? • People using the service: • Are protected from abuse, or the risk of abuse, and their human rights are respected and upheld • That is because providers who are compliant with the law will: • Take action to identify and prevent abuse from happening in a service • Respond appropriately when it is suspected that abuse has occurred or is at risk of occurring • Ensure that Government and local guidance about safeguarding people from abuse is accessible to all staff and put into practice • Make sure that the use of restraint in a way that respects dignity and protects human rights, and where possible respects the preferences of people who use services • Protect others from the negative effect of any behaviour by people who use services People focused Outcome Based

41. Registration: the cycle Registration application Information capture Judgement on risk Application made Information analysis Application assessed Judgement published Judgement made Regulatory judgement Regulatory response Ongoing monitoring of compliance

42. Information capture and analysis Information capture Information analysis

43. Making sense of shed-loads of data 43

44. The quality and risk profile (QRP) • QRP is a way of gathering all we know about organisation so as to assess risk that organisations are failing to comply with registration standards • And thus prompt front line regulatory activity and allow the judgements of this activity to be made robustly and add to the knowledge base • It is not a rating, ranking, league table or judgement in and of itself • Inspectors make judgements about compliance not the QRP • In other words the information and risk estimate are “tin-openers” not dials • Critically, it is constantly updated builds over time and is never “perfect” • We will share as much as possible with organisations themselves, we will also consider what and how we can publish at least some of the QRP

45. What sort of information do we have • Hospital episode statistics – up to 240 items of data on every admission into hospital in England • Mental health minimum data set – additional relevant info on mental health care • Annual national patient and staff surveys (in excess of 100,000 respondents each) • Clinical audit data for selected diseases • Other organisations’ data – e.g monitoring of c.diff and MRSA incidence • Notifications of incidents to the National Patient Safety Agency • ‘Soft’ local intelligence based on what patients, carers, their representative groups and staff say about care • In total about 650 separate items of data enter the model from around 95 data sets from around 30 sources

46. How do we analyse it Don’t make judgements from the data – spot outliers and ask questions

47. Observed mortality significantly different than expected Statistically relevant variation Outcomes for patients admitted with heart valve disorders

48. Combined to show “patterns of oddness” Detail

49. Alert signalled Down when a patient survives Plot goes up when there is a death Plot can never fall below zero Statistically relevant variation in real time Graph courtesy of Dr Foster Unit at Imperial College