Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health

1. Public Service Productivity Measurement: Macro or Micro? Aileen Simkins, Department of Health Co-Director of the Atkinson Review

2. What I will Talk About • Some concepts • Macro productivity measurement: the Atkinson Review (and Gershon) • Micro productivity measurement: some NHS examples • Challenges for the future

3. Productivity: What do we mean? Technical efficiency allocative efficiency value for money effectiveness economy cuts Increase in output divided by increase in volume of input Getting more for the same, or the same for less Improvingservices while money is tight

4. Measuring Productivity: Who By, What for? The XX industry is inefficient by international standards Evidence of use of resources Do you get what you pay for? Accountability to the tax payer Best of breed have 3 times more throughput than bottom quartile Benchmarking, competitive pressures Just try harder Performance management Can’t weget another one in? - Local drivers to expand/ improve

5. Some key questions • CanGovernment measures of public service productivity be valid? • Is measuring productivity the same as measuring performance? • Improving productivity matters – is measurement a side alley? • Does measurement of efficiency create distortions?

6. The Atkinson Review How to use National Accounts to measure public service productivity and How to do it better

7. How National Accounts Measure Public Services • Traditionally, output = input • From 1997, System for National Accounts changed: measure outputs via activities • UK early implementer – health (cost weighted activity index), schools (pupil days and quality adjustment), social services, social security admin • ONS began publishing productivityarticles

8. ONS Output Measure, 2003

9. Better Measures for the National Accounts • Queries on basis of public service measures • National Statistician set up Review Dec 2003, chaired Sir Tony Atkinson • ‘Measurement of Government Output and Productivity for the National Accounts’

10. Atkinson Report Jan 2005 • 9 principles ‘output should be measured in a way that is adjusted for quality, taking account of the attributable incremental contribution of the service to the outcome’ • Specific recommendations for improvement to measures used in NA • Encouraged ONS productivity articles • UK Centre for Measurement of Government Activity

11. ONS Health Productivity Oct 2004

12. DH Press Release Oct 04 John Reid (Secretary of State for Health) says “ it is absurd to measure NHS output without taking account of quality”

13. Quality as part of NHS Output • How many domains of quality? • Health gain • Patient experience • What can we measure? • How can we link quality measures to the NHS output index? • How should we weight different aspects of quality? • How valid is a partial story?

14. Accounting for Quality Change Average over last 5 years: • Value of health 1.5% • Value weight for statins 0.81% • York/NIESR adjustment 0.17% • Patient experience* 0.07% • Blood pressure control * 0.05% • Heart attack survival 0.01% Total ** 2.68% Quality adjusted output growth 6.29%

15. ONS Health Productivity 2006

16. Education and Other Areas • DfES action following Atkinson – new use of pupil attainment tests and GCSE grades • DfES article, ONS education productivity article • Controversial issues – grade drift, ‘real earnings’ effect • Also productivity articles for adult social care, social security administration

18. Value and Validity of Atkinson-based Productivity Measures • Designed to compare total outputs with total inputs • Focus on attributable impact on outcomes and quality change • Data incomplete; biased towards areas of attention / improvement; analysis by Depts • Major developmental issues on techniques • ONS consultation document Sept 2006

19. Performance = Productivity? • A perfect Atkinson measure captures all activities, all contributions to outcomes • Is this the same as measuring organisation performance? • If not – choices about key priorities, value judgements • Should those values inform the aggregate productivity measure?

20. Another macro measure: Gershon • Gershon includes savings from better procurement, Atkinson doesn’t • Specific changes in use of inputs valued as £m • Better use of productive time – evidence from outputs and outcomes?

21. Gershon and Atkinson • Gershon efficiency improvements are managed, not just measured • Strong delivery support for Gershon, changes evident • Measured to deliver target? – NAO scrutiny • Will measured productivity gains match? – HMT, OGC and UKCeMGA should be able to explain

22. Micro Measures: NHS • NHS Institute for Innovation and Improvement, with DH • Developing and issuing key metrics for health authorities, PCTs, Trusts, FTs • Systematic focus on key areas for improvement – volume, costs, variation • Clinical engagement, benchmarking

23. First Metrics: 22 Sept 2006 • Pdfs and web based tool • Finance, workforce, procurement • Clinical productivity: • Potential bed days saved through reduced length of stay • Day case rate • Reduce pre-operative bed days

24. Reduce wasted bed days 390,000 bed days £100m

25. Benchmarking • Evidence based assessment of performance • Addressing variations and helping outliers • Providing a tool kit • Opportunity to test performance against peers • Stimulus to learn how to improve practice • Recognition of success and challenge to do better

26. XXX NHS Trust

27. Value and Power of Micro Productivity Indicators • Practical engagement in what to change • Service based, clinical focus • Small steps on long journey • Risks that change happens, savings don’t • Check for unintended perverse incentives • Tipping points for cutting overheads

28. Impact of Micro Changes on Macro Measure NHS III says: 1 – Reduce Avoidable Emergency Admissions 2 – Reduce Unnecessary Outpatient Appointments, Follows-up, and DNAs 3 – Avoid Unnecessary Procedures 4 – Improve Day Case Performance 5 – Reduce Wasted Bed Days 6 – Accurate Clinical Coding 7 – Reduce Variation in Length of Stay 8 – Improve Staff Productivity 9 – Actively Manage Staff Costs

29. If NHS follows NHS III advice.. • Lower growth in some hospital activities • Shifts in unit costs • Shifts in NHS spending to primary care, community, public health (harder to count) • Probably, slower growth in output • Measurable change in quality, outcomes? • Productivity measure?

30. Measures, not Targets • Macro and micro productivity measures both need careful interpretation • Micro are more useful for managers, front line, actionability • Macro should help with ‘do you get what you pay for’ – but may mislead • Use of any efficiency measures changes behaviours – intended, maybe also unintended

31. Lessons of History ‘For the first decade of its existence (1982-92) the HCHS efficiency index received little adverse comment either from the NHS or policy analysts. However, once it was converted from a retrospective analytical exercise into a management tool it attracted a barrage of criticism….attaching rewards to counts of particular activities led to reporting drift, reclassification and at worst statistical dishonesty’ Clive Smee ‘Speaking Truth to Power’ 2005

32. Dilemma: Measuring with Right Impact • Productivity is Govt business: accountability • Many drivers in public service to improve productivity • Accountability needs metrics • Metrics are almost always partial, may mislead or have unintended incentives • Choice of metrics affects what is delivered • Moral: use mix of macro and micro indicators, with intelligence & transparency