450 likes | 580 Views
DISMEVAL A STUDY ON DISEASE MANAGMENT EVALUATION IN EUROPE BERT VRIJHOEF PH.D., PROFESSOR OF CARE FOR THE CHRONICALLY ILL INTEGRATED CHRONIC DISEASE MANAGEMENT FORUM, DEPARTMENT OF HEALTH, 29TH AUGUST 2011, MELBOURNE. A few facts. The Netherlands. Australia. 7,7 million sq km
E N D
DISMEVALA STUDY ON DISEASE MANAGMENT EVALUATION IN EUROPEBERT VRIJHOEF PH.D.,PROFESSOR OF CARE FOR THE CHRONICALLY ILLINTEGRATED CHRONIC DISEASE MANAGEMENT FORUM,DEPARTMENT OF HEALTH, 29TH AUGUST 2011, MELBOURNE
A few facts The Netherlands Australia 7,7 million sq km Canberra (capital) 22,6 million inhabitants Infant mortality rate: 4.7/1000 Life expectancy: 79.3 yrs for men and 83.9 yrs for women GDP: 1.3 trillion US dollars www.state.gov • 41,526 sq km • Amsterdam (capital) • 16,6 million inhabitants • Infant mortality rate: 4.4/1000 • Life expectancy: 78.3 yrs for men and 82.3 yrs for women • GDP: 843 billion US dollars www.state.gov
A few facts The Netherlands Australia Tour de France 2011 • Hockey Champions Trophy 2011
A few facts The Netherlands Australia Planking • Planking
Contents • Disease Management • Disease Management Evaluation • DISMEVAL, the project • DISMEVAL, the Netherlands
Take home messages • ‘Disease management’ presents a spectrum of chronic care improvement programmes ranging from no till substantial attempt to redesign practice • Measuring disease management programme performance at the population level in a scientifically sound fashion while being practicable for routine operations remains a challenge • DISMEVAL illustrates that other designs (than RCT) constitute valuable designs for the evaluation of complex interventions and provide in-depth insights • Future evaluation of ‘disease management’ should focus on the identification of subgroups and target these subgroups
DISEASE MANAGEMENT • ‘Disease management’ is a global buzzword in health care and it is often used by different people to mean different things • Structured ‘disease management ‘has been proposed as a means to improve quality and reduce the costs of health care and to improve health outcomes for the chronically ill • ‘Disease management’ presents a wide spectrum of chronic care improvement programmes
DISEASE MANAGEMENT • Coleman et al. [2008] distinguish programmes by the degree to which they work to redesign the patients’ medical care
Deficiencies in current health care systems: • rushed practitioners not following evidence based guidelines • lack of care coordination • lack of active follow-up to ensure the best outcomes • patients inadequately trained to manage their illness DISEASE MANAGEMENT
Why system change? • to put emphasize on system instead of only on physician • to categorize characteristics of successful interventions in an useful way • to appreciate commonalities across chronic conditions DISEASE MANAGEMENT
Contents • Disease Management • Disease Management Evaluation • DISMEVAL, the project • DISMEVAL, the Netherlands
DISEASE MANAGEMENT EVALUATION • What we know about the impact of interventions to manage chronic disease(s) tends to be based, mainly, on small studies that frequently focus on high risk patients, and are often undertaken in academic settings (Bodenheimer, Wagner and Grumbach, 2002). • The effects of large, population-based programmes and initiatives are less well understood (Mattke, Seid and Ma, 2007). • Four examples: Steuten et al. (2006); Mattke et al. (2007); Coleman et al. (2008, 2009); Peikes et al. (2009).
DISEASE MANAGEMENT EVALUATION • Steuten et al. (2006) found that a link between aims of disease management and evaluated indicators does not exist in a substantial part of published studies on disease management of diabetes and asthma/COPD, especially when efficiency of care is concerned.
DISEASE MANAGEMENT EVALUATION • Mattke et al. (2007) report that the evidence on the role of disease management in reducing utilization of health service is inconclusive. Support for disease management “is more an article of faith than a reasoned conclusion grounded on well researched facts”.
DISEASE MANAGEMENT EVALUATION • Coleman et al. (2008) found that the degree to which disease management redesigns practice predicts more positive outcomes. The impact of the Chronic Care Model on health care costs and revenues remains uncertain and probably varies by condition (2009).
DISEASE MANAGEMENT EVALUATION • Peikes et al. (2009) conclude that care coordination, as practiced by the programs participating in the demonstration from 2002 to 2006, holds little promise of reducing total Medicare expenditures for • beneficiaries with chronic illnesses.
DISEASE MANAGEMENT EVALUATION • The overall results of disease management evaluation studies are hard to interpret. • This is in part because of the variation in interventions, their constituting components, and the lack of standardization of initiatives (e.g. design, scope, scale, operational detail and providers involved vary widely). • Another challenge is the lack of widely accepted evaluation methods to measure and report programme performance at the population level in a scientifically sound fashion and that are also practicable for routine operations.
Contents • Disease Management • Disease Management Evaluation • DISMEVAL, the project • DISMEVAL, the Netherlands
DISMEVAL • Aim: to support the evaluation of disease management by identifying and validating evaluation methods and performance measures for disease management programmes or equivalent approaches and to make recommendations to policymakers, programme officials and researchers on best practices that are both scientifically sound and operationally feasible. • DISMEVAL brings together a multi-disciplinary team of 10 partners in 7 EU countries. It is funded under the European Commission’s 7th Framework Programme, theme Health and runs from 1 January 2009 to 31 December 2011.
DISMEVAL • Evaluating chronic disease management in Europe -- A review of evaluation methods and performance measures in use by Conklin A, Nolte E, Vrijhoef HJM (submitted) 1/3 • Review on approaches to chronic disease management in 12 European countries and data on methods and metrics to evaluate these approaches • Austria, Denmark, England, Estonia, France, Germany, Hungary, Italy, Latvia, Lithuania, Netherlands, Switzerland • key informants per country were identified and asked to collect data on: (i) chronic disease management approach and (ii) evaluation categories as defined in template • data synthesis: narrative approach
DISMEVAL • Evaluating chronic disease management in Europe -- A review of evaluation methods and performance measures in use by Conklin A, Nolte E, Vrijhoef HJM (submitted) 2/3 • most chronic disease management approaches underwent some form of evaluation, except in Latvia and Lithuania • nature and scope of evaluations varied considerably • mix of controlled and non-controlled studies measuring predominantly clinical process measures, patient behaviour and satisfaction, cost and utilisation • effects were usually observed over 1-3 years on a population of patients with a single, commonly prevalent, chronic disease
DISMEVAL • Evaluating chronic disease management in Europe -- A review of evaluation methods and performance measures in use by Conklin A, Nolte E, Vrijhoef HJM (submitted) 3/3 • majority of approaches were targeting diabetes and/or cardiovascular disease(s) • most commonly aimed to assess the performance of a given approach with rarely specified information about targets against effects might be compared • about half of evaluations used non-experimental designs without means for comparison • large differences in indicators of effect, length of observation • evaluations are carried out by different actors
DISMEVAL • Disease management evaluation – A comprehensive review of current state of the art by Conklin A, Nolte E (RAND 2010) 1/3 • aim: to help advance the methodological basis for chronic disease management evaluation by providing a comprehensive inventory of current evaluation methods and performance measures, and by highlighting the potential challenges to evaluating complex interventions such as disease management • two-stage literature review: (i) methodology papers, (ii) evaluation of disease management • 111 papers (89 journal articles, 6 book chapters, 10 working papers, 6 other)
DISMEVAL • Disease management evaluation – A comprehensive review of current state of the art by Conklin A, Nolte E (RAND 2010) 2/3 • challenges identified are conceptual, methodological, and analytical in nature • conceptual: clarification of characteristics of disease management intervention and the selection of evaluation measures • methodological and analytical: the establishment of the counterfactual is a key challenge -- in the context of multi-component, multi-actor disease management initiatives, RCTs are frequently not applicable because randomisation is not possible (or desirable) for reasons such as cost, ethical considerations, generalisability, and practical difficulties of ensuring accurate experimental design.
DISMEVAL • Disease management evaluation – A comprehensive review of current state of the art by Conklin A, Nolte E (RAND 2010) 3/3 • as alternative strategies become less of a controlled experiment, there are more threats to the validity of findings from possible sources of bias and confounding which can undermine the counterfactual and reduce the utility of the evaluation • other analytical challenges: selection of suitable control group, determining statistical power, case mix, etc. • a clear framework of the mechanisms of action and expected effects that draws on an understanding of the characteristics of disease management, those of the intervention and target populations, an adequate length of observation to measure effects and the logical link between performance measures and the intervention’s aims, elements and underlying theory driving the anticipated behaviour change
Contents • Disease Management • Disease Management Evaluation • DISMEVAL, the project • DISMEVAL, the Netherlands
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • most experience in disease management approach in NL is related to care for people with diabetes • disease management is organized by ‘care groups’: provider networks based in primary care • insurers pay a single fee to care groups, to cover an integrated bundle of care for a specific chronic disease over a fixed period of time (usually one year) • minimal required services are stipulated in national care standards
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • aim: to advance current methods of disease management evaluation by testing and validating potentially valuable research designs on data from existing programmes set in the Netherlands • assumption: advanced methods are necessary to go beyond a ‘grand mean effect of disease management’ and gain insight into what (combination of) components of the approach work(s) and for whom. • use was made of data from 9 care groups, already collected by Dutch National Institute for Public Health and the Environment, and an other 9 care groups retrospectively retrieved for this study
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • experimental comparisons are not feasible, as the DM approach to T2DM care has been dispersed across all regions of the Netherlands • two research designs were applied, i.e. meta-analysis and meta-regression, which are assumed to be particularly suitable for evaluating a heterogeneous care strategy that differs across settings in terms of the specific interventions being offered, targeted patients, and level of implementation • included outcomes are restricted to clinical endpoints, as data on more patient-centered measures, such as quality of life or satisfaction with care, as well as on the costs of care could not be retrieved retrospectively.
>8000 2000-8000 <2000 ●●● Frontrunner groups ●●●Non-frontrunner groups DISMEVAL: The Netherlands • 18 care groups • data of over 105,000 diabetes patients • baseline 8-12 months and follow-up 12 months
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • meta-analysis of individual patient data (IPD), i.e. the raw individual level data for each study, which is considered by some as the ‘gold standard’ of systematic review (Thompson & Higgins, 2005; Stewart, Tierney & Clarke, 2011). We applied meta-analysis methods to determine the impact of the programmatic approach on IPD concerning changes in eight clinical outcomes between baseline and follow-up. • meta-regression can examine multiple individual and group level factors together, the results of which may facilitate stratified medicine, and adjust for baseline factors
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • Main results from meta-analysis:
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • subgroup analysis were performed on patient, care group and process characteristics. Most notably: • DM is more effective for patients who have poor baseline clinical values (HbA1c, LDL, SBP) than for those with better baseline health • impact of DM for BP is greater in patient in higher age categories • care group characteristics did not reduce heterogeneity • patients whose cholesterol was measured more frequently show better results
DISMEVAL: The Netherlands • Main results from meta-regression:
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • meta-regression revealed that most variance exists within groups • no evidence was found for relationship between care group characteristic on any clinical outcome • process characteristics: as the duration of care increases, the positive effects of DM diminish; higher measurement frequency is associated with better outcomes • patient characteristics: impact of DM is greater as patients’ baseline values are poorer and this is more determinative than other characteristics
DISMEVAL: The Netherlands Going beyond the ‘grand mean’ – Advancing approaches to disease management evaluation in the Netherlands by Elissen A, Duimel-Peeters IGP, Spreeuwenberg C, Vrijhoef HJM • from a simple pre-post comparison one would conclude that DM does not achieve the intended goals • from mete-regression and meta-analysis it was found that the variance within care groups is much higher than between care groups • the poorer patients’ baseline values of a particular endpoint are, the more beneficial frequent measurement of that outcome is • in other words: intensive DM programs targeting patients at high risk of diabetes complications have great potential for cost-effectiveness
Take home messages • ‘Disease management’ presents a spectrum of chronic care improvement programmes ranging from no till substantial attempt to redesign practice • Measuring disease management programme performance at the population level in a scientifically sound fashion while being practicable for routine operations remains a challenge • DISMEVAL illustrates that other designs (than RCT) constitute valuable designs for the evaluation of complex interventions and provide in-depth insights • Future evaluation of ‘disease management’ should focus on the identification of subgroups and target these subgroups
THANK YOUH.J.M.VRIJHOEF@UVT.NLINTEGRATED CHRONIC DISEASE MANAGEMENT FORUM,DEPARTMENT OF HEALTH, 29TH AUGUST, MELBOURNE