240 likes | 251 Views
This talk discusses the problem of deriving a minimally important difference (MCID) for PRO (Patient Reported Outcomes) data. It explores statistical methods and provides a worked example on menopause symptoms. Open issues and suggestions for deriving MCID between treatments are also discussed.
E N D
Deriving the minimally important difference for PRO data EFSPI HTA Scientific Meeting Berlin, 2014-09-25 Dr. Christoph Gerlinger
Acknowledgements • This talk is based on joint work with my colleagues • Florian Hiemeyer • Dr. Thomas Schmelter <2> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Outline • Problem statement • Statistical methods to derive a minimally important difference (MCID) for the patient • Worked example • Points to consider in deriving an MCID for the patient • Open issue: MCID between treatments <3> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Problem statement • For every measure we need an interpretation what the number does mean • 50 kg body weight • 4.9 mmol/l of cholesterol • 14 mm difference in pain of on a 100 mm visual analogue scale • Baseline • End of study <4> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Problem statement – 2 • For body weight or cholesterol this is a clinical judgment • Statistics can’t help • For pain we can ask the patient what the measure does mean to her/him • And use some statistics to derive the minimal important difference for the patient • Use the minimal important difference for the patient to derive a responder definition • Open problem: How to derive the minimal important difference between different treatments • Suggestions very welcome: christoph.gerlinger@bayer.com <5> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Methods to derive MCID for PROs • Anchor based with patient satisfaction as anchor • Discrimant analysis • ROC analysis • Mean of patients who changed (Juniper et al. 1994) • Distribution based • Half standard deviation rule (Norman et al. 2003) • Standard error of measurement (Wyrwich et al. 1999) <6> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
FDA Guidance (Dec 2009) • www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf (2014-09-03) <7> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Worked example - menopause • Menopause: the permanent cessation of the primary functions of the human ovaries • Typically occurs in women in midlife, during their late 40s or early 50s, and signals the end of the fertile phase of a woman's life. • The menopause is a natural and irreversible process, it is not a disease. • The decline in estrogen results in a range of symptoms, e.g. vasomotor symptoms (hot flushes, night sweats and palpitations), insomnia, mood changes, loss of libido, etc. • These symptoms can have a significant effect on a woman’s quality of life. <8> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Data from a clinical trial • 735 patients randomized (1:1:1:1) to 3 active doses and placebo for 12-weeks • minimum of 50 moderate to severe hot flushes per week • Moderate hot flush: heat sensation with sweating • Severe hot flush: sweating so intense cause interruption of current activity • primary efficacy variables were mean changes from baseline to weeks 4 and 12 in the weekly frequency and weekly mean daily severity of moderate to severe hot flushes (as per FDA guidance) • hot flushes recorded daily on diary cards • Archer DFet al. Menopause. 2014 Mar;21(3):227-35. • MCID analysis on clean database but withouttreatmentinformation • Results of blindedanalysessentto FDA • FDA Type A meetingpriortounblindingtoagree on methodsandresults <9> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Patients‘ view Note: low proportion of no change/worse due to several active arms and pronounced placebo effect 1 Guy W (Ed). ECDEU Assessment Manual for Psychopharmacology. 1976. Rockville, MD, US Department of Health, Education, and Welfare. <10> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Change in hot flushes by aggregated CGI <11> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Density estimates for hot flushes by aggregated CGI <12> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
MCID: Anchor based approach • According to non-parametric discriminant analysis • the cut-off between “no change/worse” and “minimally improved” was -19.1 moderate to severe hot flushes per week • this cut-off value can be interpreted as the minimal clinically important difference (for the patient) • The cut-off between “minimally improved” and “much/very much improved” was -40.3 moderate to severe hot flushes per week • this cut-off value can be interpreted as a clinically important difference (for the patient) <13> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
ROC (no change or worse vs. minimally improved) <14> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
MCID: Distribution based approach • According to Norman et al. (2003), a MCID for a patient reported quality of life outcome can be calculated by using half the standard deviation of the population • The resulting MCID is 19.2 hot flushes per week. • Compares well to the MCID of 19.1 for the anchor based approach <15> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Summary of worked example • MCID of the patients could be derived empirically • We translated the MCID of the patients into a responder definition: a subject is a responder if and only if she has • a reduction of at least 19.1 moderate to severe HF per day at week 4 AND • a reduction of at least 40.3 moderate to severe HF per day at week 12 • Confidence interval for MCID can be derived by bootstrapping • Methods and results were discussed and agreed upon with regulator before unblinding the trial • Details in Gerlinger et al. 2012 <16> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Points to consider – anchor question • Choice of anchor question • Many anchor questions available • Makes little difference (work in progress) • Satisfaction with treatment influenced by efficacy and safety <17> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Points to consider – anchor method • Choice of statistical method • Discriminant analysis • ROC curve • Mean of improved <18> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Points to consider – distribution based <19> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Summary - points to consider • Several methods to derive the patients’ MCID • No clearly best method • One bad method (mean of improved patients), in my view • Results of different methods are often close • Patients’ MCID can be transformed into responder definition • The empirically derived MCID and responder definition were acceptable to stakeholders in several indications • Hot flushes, pain, acne <20> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
MCID between treatment arms • So far, we derived the MCID of a patient • Question from a stakeholder: MCID between treatment arms? • Quite obvious to me • MCIDpatientMCIDbetween treatments • But not so obvious whether there is a strict inequality • If 15 mm in pain reduction make a difference to patients • Are two treatments with a 30 mm and a 40 mm reduction equivalent or different ? • Judgment ? <21> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
One suggestion • define the MCID between treatments in terms of the difference in response rates • You can then “reverse engineer” the MCID on the original scale • Yes, but • A 5%-point difference in the rate of patients satisfied with their acne treatment is not the same as a 5%-point difference in mortality, in my view • Again, this is a judgment <22> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Conclusion • The patients’ MCID can be derived for a PRO using statistics • This is better than arbitrarily choosing a nice cut-off like 3.1415 or 2.7182 or log2(1.4142) • The MCID can be used to derive a responder definition • No method to derive MCID between treatment arms, yet • Your ideas, please <23> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
References (recommended reading) • Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003;41:593–596. • Gerlinger C, Schmelter T. Determining the Non-Inferiority Margin for Patient Reported Outcomes. Pharmaceutical Statistics, 2011 Sep;10(5):410-3. • FDA PRO Guidance. www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf (2014-09-03) • Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific Quality of Life Questionnaire. J Clin Epidemiol. 1994 Jan;47(1):81-7. • Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J Clin Epidemiol. 1999 Sep;52(9):861-73. • Gerlinger C, Städtler G, Götzelmann R, Graupe K, Endrikat J.: A non-inferiority margin for acne lesion counts. Drug Information Journal. 2008 Nov;42(6):607-15. • Gerlinger C, Schumacher U, Faustmann T, Colligs A, Schmitz H, Seitz C. Defining a minimal clinically important difference for endometriosis-associated pelvic pain measured on a visual analog scale: analyses of two placebo-controlled, randomized trials. Health Qual Life Outcomes. 2010 Nov 24;8(1):138 • Gerlinger C, Gude K, Hiemeyer F, Schmelter T, Schäfers M. An empirically validated responder definition for the reduction of moderate to severe hot flushes in postmenopausal women. Menopause. 2012 Jul;19(7):799-803. • Archer DF, Schmelter T, Schaefers M, Gerlinger C, Gude K. A randomized, double-blind, placebo-controlled study of the lowest effective dose of drospirenone with 17β-estradiol for moderate to severe vasomotor symptoms in postmenopausal women. Menopause. 2014 Mar;21(3):227-35. <24> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25