Advanced analytics in health and social care

1. Advanced analytics in health and social care OR Society: Developments in advanced analytics and big data 30 April 2014 Cono Ariti Senior Research Analyst The Nuffield Trust

2. The Nuffield Trust • Promote independent analysis and informed debate on healthcare policy across the UK • Charitable organization founded in 1940 • Formerly a grant-giving organization • Since 2008 we have been conducting in-house research and policy analysis • Significant interest in uses of advanced analytics William Morris 1st Viscount Nuffield (1877 -1963)

3. Nuffield Trust Research team projects Risk sharing for CCGs WSD Combined predictive model Person based resource allocation Virtual Wards Social care at end of life Marie Curie Nursing Service Cancer and social care Predicting social care costs Integrated care pilots nuffield trust nuffield trust nuffield trust nuffield trust nuffield trust nuffield trust nuffield trust nuffield trust nuffield trust nuffield trust

5. History of UK NHS spending

6. Source: Department of Health

7. Health and social care timeline – an individual’s history

8. Data are everywhere… Ambulance Control NHS Direct Up there Commissioning data ... Commissioner Pharmacy Community Health Services A&E GP IP OP Social care provider Housing Council Tax Council Social Services Local Authority

9. Health and Social Care Data

10. Information flows for typical research project

11. Why rely on using existing data for research?

12. Used of linked person level data • Audit and Quality Improvement • Patient safety (e.g. monitoring drug side effects or surgical mortality rates) • Public Health programmes (immunisation; monitoring cancer rates) • Planning services (e.g. ICU bed availability; pandemic flu plans; manage changing patterns of demand) – predictive modelling • Resource allocation • Evaluate Services (are they effective and cost effective?) • Manage Performance (e.g. readmission targets; health outcomes indicators) • Research

13. Predictive modelling

14. Uses of predictive risk techniques

15. Predictive modelling used to target interventions Top 0.5% Case Management 0.5 – 5.0% Intensive Disease Management 6 - 20% Less Intensive Disease Management Wellness programmes 21 – 100% Adjust intervention depending on risk strata

16. Introduction of predictive modelling to UK • Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143 • Patterns in routine data identify high-risk people next year • Relies on exploiting existing information • Use pseudonymous, person-level data • In health sector a number of predictive models are available e.g. PARR++ and the combined model.

17. PARR-30: a predictive model for readmission within 30 days

18. Why predict readmissions within 30 days? • Readmissions are costly, suboptimal health care - costs to the NHS estimated at £1.6 billion each year • DH guidance for the NHS proposes commissioners do not pay provider hospitals for emergency readmission within 30 days of a selected index elective admission • Rate of readmissions will also play an important part in monitoring health system performance, as one of the new English Public Health “outcome indicators”

19. How is PARR30 different from PARR++? PARR30 Hospital provides SUS Patient nears discharge CCG runs PARR++ Risk score calculated on ward Patients selected for intervention (via GP) Any extra intervention put in discharge plan Predicts readmission in next year Predicts readmission in 30 days

20. Model development Hospital of current admission Patient age Deprivation (via post code) History of emergency admissions: Current? Last 30 days? Past year? History in the prior two years of eleven major health conditions drawn from the Charlson co-morbidity index

21. Results • The performance of the model was respectable, with a positive predictive value (PPV) of 59.2% and area under the ROC curve (“c-statistic”) of 0.70. • For the higher-risk patients (risk score > 50%), readmission rates ranged from 47.7% up to 88.7%. However, these patients only represented a small share (1.1%) of all patients analysed. Receiver Operating Characteristic Curve (ROC) for the bootstrapped central estimate (red line) and 95% confidence Intervals (shaded area)

22. Resource allocation

23. Person Based Resource Allocation A resource allocation formula at general practice level based on individual level characteristics (person-based resource allocation)

24. Reviews of resource allocation in English NHS Hospital and Community Health Services , 1976- today Drawn from Bevan, and Bevan and Van der Ven Note: RAWP = Resource Allocation Working Party RoR = Review of RAWP AREA = Allocation of Resources to English Areas CARAN = Combining Age Related Additional Needs

25. Resource allocation in the NHS, England

26. Person based resource allocation                                                                                                     Practice population                                 General population @ £10,000 per person per year @ £1,000 per person per year @ £100 per person per year

27. Costsi= f(needsi, needsa, supplya)

28. Modelling • Models predict hospital-based expenditure excluding maternity and mental illness • Explanatory variables from prior years included: • Age and sex (36) • Diagnostic categories from hospital utilization in previous years (152) • Attributed GP and small area needs characteristics (135) • Attributed small area supply characteristics (63) • Primary Care Trust (PCT) to capture local commissioning trends (152) • Note: did not consider variables with potentially adverse incentive effects, e.g. number of encounters

29. Results from testing various models predicting costs for 2007/08 using data from 2005/06 & 2006/07

30. Results applied in acute component of the Fair Shares Toolkit http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_111057

31. Evaluation of complex interventions

32. Need to know what works In a practical setting – “real world evaluation” Clarify the debate Likely impacts – unbiased results Link to qualitative work Refine programs Obtain feedback and learnings – the pain of implementation Explore sub-groups – where did it work? Where could it work? Need for evaluation

33. Randomised control trials “Gold standard” May not be feasible or ethical Inclusion and exclusion rules can limit generalisation Are still subject to poor implementation – can induce bias Expensive Observational studies Typically no “natural” experiment exists Often no comparable control group to provide a fair assessment Issues with evaluations

34. Conceptual framework: Rubin causal model • There are two (or more) potential outcomes for each individual i: • Yi1 Outcome if treated • Yi0 Outcome if not treated • The causal effect we want to estimate is: • Sample (or Population) Average Treatment Effect = E{Yi1– Yi0} • where this average is over all individuals in the sample (or population) • but only one of Yi1, Yi0 is known.

35. How far can we get without randomisation? One approach – make the strong ignorability assumption, i.e. Yi1, Yi0╨ z | x, Where z is treatment assignment, x is the vector of observed covariates.

36. Propensity scoring • From Rosenbaum and Rubin (1983): • A balancing score, b(x), is a function of the observed covariates x such as that the conditional distribution x given b(x) is the same for treated (z=1) and control (z=0) units: • x ╨ z | b(x) • Examples: • 1. X – the observed covariates • 2. The propensity score i.e.b(x) = Pr (z = 1 | x) • Rosenbaum and Rubin show that the propensity score is the coarsest balancing score

37. Big theorem • Suppose treatment assignment is strongly ignorable and b(x) is a balancing score. Then: • E{Yi1| b(x), z = 1} – E{Yi0|b(x), z=0} = E{Yi1-Yi0|b(x)} • Matching on propensity scores “mimics randomisation” at levels defined by the propensity score assuming that propensity scores have been correctly estimated

38. Matching Demographics – age, gender, deprivation, ethnicity Prior acute care service use – admissions, OP and A&E attendances Prior diagnoses, risk score Balance In this case all matching variables Additional variables such as length of stay, additional diagnoses and longer service use history Assures comparability between the groups Variables used in matching

39. Algorithm Exact match not possible Computer intensive “genetic algorithm” Uses a weighted “distance” to determine closest match Automatically assesses balance and moves to an improved solution Assessing Balance On overall group similarity Compares means and distribution of variables in the two groups Matching Algorithm

40. Assessing balance

41. Standard statistical methods to estimate the difference in the two groups Regression models, difference in difference analysis Naturally accommodate the matching By including matching variables in the statistical adjustment remaining imbalances can be reduced – “doubly robust” Methods exist for sensitivity analysis – impact of unobserved variables Analysis of matched control studies

42. Sensitivity Analysis

43. Intervention: Nursing care support to people at end of life, in their homes Nuffield commissioned to evaluate impact: Are recipients more likely to die at home? Reduction in emergency hospital admissions at end of life? Methods: Retrospective matched control study – use of already existing administrative data Case Study: The Marie Curie Nursing Service

44. >1M individuals - died Jan 2009 to Nov 2011, did not receive service (everyone else) Matched control studies – broad aim 30,000 individuals - died Jan 2009 to Nov 2011 & received Marie Curie nursing service before death

45. >1M individuals - died Jan 2009 to Nov 2011, did not receive service (everyone else) Aim to find 30,000 individuals who match almost exactly on a broad range of characteristics Use this group as study control group Matched control studies – broad aim 30,000 individuals - died Jan 2009 to Nov 2011 & received Marie Curie nursing service before death

46. Final datasets available for analysis Nuffield trust MC data - desensitised ONS deaths Hospital inpatient, outpatient, AE Identifiers: HESID on all Use all this info to carry out matched control analysis

47. Control group – how well matched? Marie Curie Controls

48. Control group – how well matched? Diagnostic history Marie Curie Controls

49. Key Result 1: Where do people die?

50. Key Result 2: How do emergency admissions compare? Service starts