On Measuring Outcomes and Productivity in Canada’s Health Care Sector

1. On Measuring Outcomes and Productivity in Canada’s Health Care Sector • first principles and basic definitions • the UK Atkinson Report, the System of National Accounts (SNA), and the “standard” approach to measuring (health sector) productivity • empirical nuggets and “awkward facts” • alternative and better approaches NB – More detail can be seen in the “notes view”; still draft, please do not circulate without permission Michael Wolfson Statistics Canada

2. First Principles - I • Population health is the fundamental objective • Health outcomes relate to / depend on health interventions • i.e. social activities, whether deliberate or inadvertent • Health care is one kind of intervention which often contributes to health • but not always, and certainly not solely

3. First Principles - II • Doing more with less is a good thing • i.e. being more efficient or more productive is beneficial • People and care providers are heterogeneous • so summing or averaging to produce overall indices can produce misleading results

4. Definition (by Construction) - Population Health I • ask everyone (or a sample thereof) a structured set of questions (or do an exam) to assess each person’s health status • i.e. a profile for each person (n.b. gives micro detail) • construct an index for each person, based on their health profile • e.g. McMaster Health Utility Index, or QALY • average over people (perhaps age-standardized)

5. or combine individual-level summary health indices with life table (mortality rates) to measure Health-Adjusted Life Expectancy (HALE) Definition - Population Health II

6. “Cause – Deleted” Changes in Life Expectancy (LE) and Health-Adjusted Life Expectancy (HALE) LE HALE Source: Manuel et al, ICES and Health Canada, NPHS

7. health status “before” health status “after” health intervention other factors Definition - Health Outcome health outcome  change in health status attributable to a health intervention (for an individual)

8. Charles Wright on Vancouver Cataracts • pre- and post-surgery patient self-completed questionnaires • “31% of patients booked for cataract surgery report a visual function score of 91 points or more on a scale of 100. … • “These data tend to confirm the observation that cataract surgery is now occurring in many patients with minor degrees of self-reported visual disability. … • “The overall results are positive, but 27% of patients show either no change or deterioration of VFA (Visual Function Assessment) score after the operation.”

9. 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0 5.5 Relative Risks of “Preventive” Tamoxifen Breast Ca Fractures CHD Cataracts Stroke Deep Vein Thromb Endometrial Ca (9.27) Pulmonary Emb (Fisher et. al., J National Cancer Institute, 1999)

10. Simulated Change in Life Expectancy for Canadian Women for Alternative Scenarios of Preventive Tamoxifen (95% CIs) 0.12 0.1 0.08 0.06 Change in Life Expectancy 0.04 0.02 0 -0.02 -0.04 1.66 2.08 2.49 2.91 3.32 3.74 4.15 42.3% 24.6% 16.2% 9.0% 4.0% 2.2% 1.7% 5-yr Predicted Risk / Proportion of Women Affected (Will et. al., British J Cancer, 2001)

11. Definition – Health Care, per the Evans and Stoddart “Plumbing Diagram” “thermostat” “bottom line”

12. (Tu et al on Coronary Surgery) n.b. virtually no differences in one year survival; but no data on differences in health-related QoL e.g. almost 17x, with no benefits?

13. (Career Earnings and Death) Mortality Rate (%, age 65-70) 18 Career Earnings and Death 16 for 500,000 Canadian Men 14 12 10 8 6 4 2 0 0 20 40 60 80 100 120 Average Earnings (age 45-64, 1988 $000s) top quintile Source: Wolfson et al., Gerontology, 1993

14. Definition – Productivity (General) • productivity level  “output” / “input” • productivity growth growth in outputs - growth in inputs • i.e. getting more output for given inputs, or getting the same output from fewer inputs • n.b. in common parlance • no presumption that everything has to be measured in $$$ • indeed, usual thoughts are in physical units (e.g. patients seen or cataracts done per day)

15. Definition – Productivity (business school) • let me compare myself to another firm, typically a competitor • is she producing her widgets at lower unit costs than me? • i.e. benchmarking for individual product lines

16. Definition – Productivity (“standard” economics and SNA) • the economy has myriad productive agents (firms) • each of whom uses inputs = total capital services + total labour services (factors of production) • to produce outputs (goods and services) summing to GDP • everything is measured in $ -- with the total being (conceptually) the sum of unit prices x quantities • but over time, prices (p’s) change, and this is not “real” • and quantities (q’s) change e.g. in terms of “quality” • to measure productivity, time series of outputs and inputs are constructed • taking out “pure” price changes, and • adjusting for improvements in quality • so that  productivity =  output – sum { inputs }

17. Definition – Health Sector Productivity (“standard” economics and SNA) • “standard” economists and National Accountants want to treat “health care” as an industry, with “health care services” its outputs, analogous to private sector industries • “firms” in the health sector are divided (mainly) into hospitals, nursing homes, and providers of ambulatory care (OECD SHA) • n.b. no concept of “regional health authority” or “local health integration network” • ideal concept for “outputs” is care for “episodes of illness”, though DRGs in practice (OECD SHA) • n.b. no concept of “continuum of care”, nor chronic illness, nor recognition of co-morbidities

18. Public Sector Challenge to “Economists’ Productivity” “Profits” ??? Outputs Inputs Public Sector Commercial Sector

19. UK Office of National Statistics (ONS) and their “Productivity Paradox” • UK Labour Government massively increased spending in health care (and education) starting in the late 1990s • the ONS had been dutifully measuring public sector productivity, using SNA / economists’ concepts • productivity declined (unfortunately) • ONS (2004) asked Sir Tony Atkinson what to do • (and many jurisdictions are considering the Atkinson report recommendations, as well as gradually adopting the OECD’s System of Health Accounts)

20. ONS, Atkinson, and Productivity • mandate from ONS National Statistician: • “To advance methodologies for the measurement of government output, productivity and associated price indices” (OK) • “in the context of the National Accounts” (Oh oh!) • question: why not first pose issue in general • and then only secondarily ask whether SNA is an appropriate framework, and if not what would be?

21. Atkinson Report – Analysis I • SNA data are the essential foundation • for macroeconomic management • and as an indicator of social welfare • UK context per Bank of England (May 04) • CPI up 10% from 1997Q1 to 2003Q4 • nominal government spending up 62% • ONS measure of real public sector output up 14% (Huh?) • GDP as welfare measure • more $ on (e.g.) health care treatments increases welfare – certainly if appropriate and effective, but “asymmetric information”

22. Atkinson Report – Analysis II • “National Accounts are not a substitute for performance indicators” (para 1.27) • “It is not necessarily the case that even a crude measure of government output is preferable to an index based on total cost.” (para 2.25) – i.e. the conventional way of doing the SNA • measurement of quality change (e.g. improvements in methods and technology) is a major challenge • the UK, as part of the EU, is bound to measure SNA according to international standards (but the US and Canada, so far, have ignored these)

23. Atkinson Report – Selected Recommendations • the SNA should measure government non-market output (e.g. health care services) using a procedure parallel to that of the market sector (para 4.7) • use the “treatment” or GP visit as the canonical “output” of the health care sector • weight different kinds of treatment by their costs • try to adjust for quality change – ideally by moving from treatments to “care pathways”, and connecting care to health outcomes • n.b. sounds good, but feasibility? • also treat shorter waits as improved quality • n.b. nothing on appropriateness, or “watchful waiting”

24. (Skinner I) tractors “Technology Adoption from Hybrid Corn to Beta Blockers”, Skinner and Staiger, NBER, 2005 corn computers beta blockers corn corn

25. (Skinner II) adjusted 1 yr mortality rate adjusted 1 yr mortality rate $$$ beta blockers adjusted 1 yr mortality rate beta blockers (education and) social capital (education and) social capital

26. (Skinner III) (Source: Skinner, Staiger, Fisher; Medical Technology, 2006)

27. “Wall of Ignorance”

28. Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions British Columbia Ontario Quebec percent dead within one year Alberta percent revascularized within 30 days

29. E. A. Codman and W.E. Deming • Codman: early 1900s Boston surgeon • famous for “End Results Cards” – to keep track of surgical patients and follow them up one year later to • observe outcomes • systematically learn from experience • 100 years later: not yet implemented in health care • Deming: post WW II concern with product quality in manufacturing • father of field of statistical process quality control • 50 years later: not yet implemented in health care

30. Underlying Person-Oriented Information (POI) for Heart Attack / Revascularization Analysis one year observation window (excluded) one year follow-up window Heart Attack (AMI) Treatment (revascularization = bypass or angioplasty) Death

31. Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions British Columbia Ontario Quebec percent dead within one year Alberta percent revascularized within 30 days

32. Heart Attack Patients, 2000: Treatment and One Year Mortality Rates for Large Canadian Health Regions British Columbia Ontario Quebec percent dead within one year Alberta better (less intervention, & better survival) (more output ???) percent revascularized within 30 days

33. Hospital 65+ Patient Co-morbidity based on 676,508 hospital inpatient discharges across 10 provinces in 2001/2

34. Health Care Outputs or Health Outcomes? • SNA approach: health care inputs  health care outputs (i.e. “treatments”) • leave for others to figure out connections from health care outputs  health outcomes (para 7.27, OECD SHA) • public policy priority: what (broad) allocation of resources produces the most “health gain” (i.e. increase in population health) – inputs  outcomes • SNA approach is helpful on inputs and costs • though focus on aggregation distracts from “benchmarking”, i.e. “firm” level analyses • and SNA compulsion to create an artificial concept of “output” is useless for this purpose

35. Platitudes? You can’t manage what you can’t measure You get what you measure “Don’t ask how many (health care) events per pound; ask how much health per pound.” D. Berwick, BMJ 2005

36. access / waits Ontario Health Scorecard (Ontario Framework) clinical results continuity of care production and use of evidence health status sensible allocation healthy living spending / resources sustainability / equity

37. Vision – Coherent, Integrated Statistical System Broad Summary Indicators Health Accounts / Simulation Models Regional Indicators / Planning Info Facility Management Information / Unit Costs Basic Encounter Data / Health Surveys