The analysis of longitudinal studies in rheumatology: Is it done correctly?

1. The analysis of longitudinal studies in rheumatology: Is it done correctly? 6th Annual Clinical Research Methodology CourseFriday, December 16, 2011 Emmanuel Lesaffre Department of Biostatistics, Erasmus MC, Rotterdam, the Netherlands L-Biostat, K.U.Leuven, Leuven, Belgium In collaboration with Karolina Sikorska, Maurits de Rotte, Jolanda Luime, Mieke Hazes

2. Conclusions • Many longitudinal studies in rheumatology are not using an appropriate statistical methodology. • Repeated significance tests and summary statistics methods are inappropriate in practice and can not show the evolution of an outcome over time. • Classical repeated measurement methods are inappropriate in practice. • The mixed effects model and the GEE approach can make use of all available cases independently whether they have data on all time points studied. • The mixed effects model and the GEE approach are sparsely used in longitudinal studies rheumatology.

3. Description JIA study

4. Juvenile Idiopathic Arthritis (JIA) study • Description • Longitudinal study: 5 time points (0, 3, 6, 9, 12 months) • Recruitment at University Medical Centre Utrecht (UMCU) and Wilhelmina Children’s Hospital, the Netherlands • Children diagnosed with JIA according to the ILAR criteria and started methotrexate (MTX) therapy between 1990 and 2006 • All patients and their parents gave their informed consent and approved by the medical ethics committee of the UMCU • Treatment was tight-controlled using a standardized report form on disease activity every 3 months • Information on MTX usage, disease activity, route of administration, dosing of MTX, reasons for ending MTX treatment, concomitant therapy and laboratory parameters were collected 4

5. JIA study Description 183 JIA patients of our test cohort were genotyped for ABCB1 3435C>T. 2 groups: genotype CC homozygous (53 patients, 28%)  T-carrier (129 patients, 72%) Continuous response = erythrocyte sedimentation rate (ESR) was obtained on all 5 visits for 106/183 patients (58%): CC homozygous (31 patients, 29%), T-carrier (75 patients, 71%) = complete cases ESR was log-transformed because of a skewed distribution. Binary response = ACR30 response, obtained for all 183 patients on all visits. Of the 183 patients, 46 (25%) responded at 3 months, 85 (46%) at 6 months, 110 (60%) at 9 months and 113 (62%) at 12 months. 5

6. Classical analyses for ESR

7. JIA study Classical analysis (ESR) 7

8. Classical approachesfor comparing 2 groups over time Repeated significance tests (t-tests, Wilcoxon tests, etc) Summary Statistic Method: summarizing whole curve by e.g. AUC and compare AUC between 2 groups with t-test Repeated measurements ANOVA MANOVA 8

9. 1. Repeated significance tests

10. 1. Repeated significance testsJIA study (ESR) Here unpaired t-tests giving P= 0.066, 0.12, 0.62, 0.33, 0.44 Nowhere significant  no real issue here Suppose P= 0.066,0.03, 0.62, 0.33, 0.44 … what to conclude? Or P= 0.066, 0.12, 0.62, 0.33,0.03 … what to conclude? If no adjustment of P-value, then problem of multiple testing 10

11. 1. Repeated significance tests Popular approach in medical research But Inefficient: only patients are included that are present at visit Likely to be biased: missing data process can bias analysis No good insight in individual evolution: link between subsequent observations is lost Turns longitudinal study into several cross-sectional studies Not suitable for contemporary rheumatology studies 11

12. JIA study • All individual profiles (ESR) • CC homozygous T-carrier

13. JIA study • 3individual profiles (ESR)

14. 2. Summary Statistic MethodJIA study (ESR) Compute Area-under-the-Curve (AUC) for each profile On complete cases only: CC-Homozygous – T-carrier= -1.33, P = 0.54, 95% CI=(-5.61; 2.96) Complete the incomplete cases by LOCF: CC-Homozygous – T-carrier= -3.78, P = 0.04, 95% CI=(-7.33; -0.21) In addition one could compare Averages of the profiles …. 14

15. JIA study • 3individual profiles (ESR): incomplete profiles completed by Last-Observation-Carried-Forward (LOCF)

16. 2. Summary Statistic Method Not frequently used in medical research Recommended for balanced data Balanced: all patients are examined at all (regular) visits Difficult for unbalanced data, i.e. when there are missing observations and patients can come at all times LOCF is a popular approach to make data more balanced LOCF Imputes unrealistic values Underestimates variability of the data Not suitable for contemporary rheumatology studies 16

17. 3. Repeated Measurements ANOVA One of only 2 approaches available for analysis of repeated measurements 50 years ago Requires balanced data Patients with missing values are excluded Time points must be regular, patients with irregular time points are excluded Restrictive assumptions on correlation between subsequent responses: correlations are equal (corrections available but not sufficient) Splits treatment effect up into time, group and group*time effect Not suitable for contemporary rheumatology studies 17

18. 3. Repeated Measurements ANOVAJIA study (ESR) • On complete cases only: • CC-Homozygous – T-carrier= • Time effect: P < 0.001 • Genotype effect: P=0.47 • Time*genotype effect: P=0.02 • Complete the incomplete cases by LOCF: • CC-Homozygous – T-carrier= • Time effect: P < 0.001 • Genotype effect: P=0.04 • Time*genotype effect: P=0.34

19. 3. Repeated Measurements ANOVAJIA study (ESR) • Assumption equal correlation??

20. 4. MANOVA Other of only 2 approaches available for analysis of repeated measurements 50 years ago Requires balanced data Patients with missing values are excluded Time points must be regular, patients with irregular time points are excluded No structure allowed on correlations  large studies are required Splits treatment effect up into time, group and group*time effect Output more difficult to understand Not suitable for contemporary rheumatology studies 20

21. 4. MANOVAJIA study (ESR) • On complete cases only: • CC-Homozygous – T-carrier= • Time effect: P < 0.001 • Genotype effect: P=0.47 • Time*genotype effect: P=0.03 • Complete the incomplete cases by LOCF: • CC-Homozygous – T-carrier= • Time effect: P < 0.001 • Genotype effect: P=0.04 • Time*genotype effect: P=0.36

22. Modern analyses for ESR

23. “Modern” approachesfor comparing 2 groups over time Types of missing data processes Approaches to deal with missing data Mixed effects models GEE models 23

24. 1. Types of missing data processes • What are missing data? • Data that are not observed • How are missing data generated? • Patients/clinicians forget to fill item(s) in questionnaire • Patients refuse to fill item(s) in questionnaire • Lost or damaged biological sample • Patients miss a visit because … • Patients decide not to return to clinician anymore • Patient died • … • Different reasons why data are missing

25. Impact of missing data • What is believed • Loss of efficiency • In reality • Loss of efficiency • Biased results (often) • Classical statistical methods need to be adapted • Approach • CLEVER explicit or implicit imputation of missing data • But: can NEVER replace true data

26. Terminology • Monotone missing • Also called dropouts • Non-monotone missing • Also called intermittent missingness

27. More terminology • Missing completely at random • Missing at random • Missing not at random – informative missing • Terminology might be confusing – Rubin (1975)

28. Missing completely at random (MCAR) • Probability of missingness isindependentofall responses • Examples • A random selection of teeth in mouth are taken in the study • A blood tube is dropped • Patient died in a car accident, but careful: patient could have experienced a sleep attack when taking a dopamine agonist • Then • Simple mean of response is unbiased estimate of true mean • Classical statistical techniques (repeated t-tests, Summary statistics method, repeated measurements ANOVA, MANOVA) can be used • Impact: loss of efficiency

29. Missing at random (MAR) • Probability of missingness dependsonobserved responses • Examples • Study design specifies that if blood pressure is not lowered patient will be removed from anti-hypertensive trial • Multi-stage screening: data are missing at subsequent stages due to result at initial stage (negative test) • Then • Simple mean of response is biased estimate of true mean • Classical statistical techniques canNOT be used • Statistical tests to distinguish versus MCAR exist • Impact: loss of efficiency + bias, but likelihood analysis can correct for bias due to missingness

30. Missing not at random (MNAR) • Probability of missingness dependsonobserved responses & unobserved responses • Examples • Patient shows a flare up in the disease unobserved in the study + patient decides to leave the study • Then • Simple mean of response is biased estimate of true mean • Classical statistical techniques canNOTbe used • No test to distinguish versus MAR • There is no satisfactory analysis, ONLY sensitivity analysis • Impact: loss of efficiency + bias, only a sensitivity analysis can shed light on the problem

31. Bias of simple mean

32. 2. Approaches to deal with missing data • Prevention and planning • Analytical remedies • Complete case analysis • Available case analysis • Imputation techniques • Likelihood-based analyses • Weighted analyses • Sensitivity analyses • Most appropriate statistical solutions are COMPLICATED

33. 2. Approaches to deal with missing data • Complete case analysis (MCAR) • Default analysis in many packages, only ok for MCAR • If not MCAR: substantial bias can be the result • Available case analysis (MCAR) • Use foreach variable/time point all observations available • Single value imputation (MCAR, MAR??) • Mean value imputation • Hot decking • LOCF methods are based on unrealistic models & underestimate variance

34. LOCF time Example: LOCF

35. 2. Approaches to deal with missing data • Multiple imputation (MAR) • Explicit imputation of missing data • Incorporate random mechanism • Generate M different completed imputed data sets • Combine M means and M variances 1 overall mean & variance method is based on statistical model for imputation

36. 2. Approaches to deal with missing data • Likelihood-based models (MAR) • Implicit imputation of missing values • Model-based: • Linear & generalized linear regression • Linear & generalized linear mixed models • … method is based on statistical model for response • GEE models (MCAR) and weighted GEE models (MAR) • Bayesian models (MAR)

37. 2. Approaches to deal with missing data • MNAR models • Model also missing data mechanism • Complex modelling • Never completely satisfactory  sensitivity analysis • BUT: if time points are close to each then MNAR close to MAR

38. 3. Mixed effects models • Two examples

39. 3. Mixed effects models • Assumption in mixed effects models

40. 3. Mixed effects models Not enough used YET in medical research Recommended for contemporary rheumatology studies Allows unbalanced data None of the patients are deleted from the study Irregular time points are allowed No explicit imputation of responses Time evolution, effect of covariates and correlation structure: all can be flexible and modelled BUT One needs statisticians for the job Luckily: they are OFTEN cheap 40

41. 3. Mixed effects models Linear mixed effects models Response = continuous Normality assumptions Generalized linear mixed effects models Response = continuous or discrete Normality assumptions 41

42. 3. Mixed effects modelsJIA study (ESR) On ALL cases: CC-Homozygous – T-carrier= Time effect: P < 0.001 Genotype effect: P=0.03 Time*genotype effect: P=0.09 + ESTIMATES and 95% CIs 42

43. 3. Mixed effects modelsJIA study (ESR)

44. 4. GEE models • Can cope only with MCAR • Makes only few assumptions about distribution of data • Most popular for discrete responses • Can also be used for continuous response

45. 4. GEE modelsJIA study (ACR30) • Binary response • No missing data in the study • Possible analyses • Chi-square tests (Fisher’s exact tests)at each time point • Generalized mixed effects model • GEE model • Same comments as before • Here all analyses gave non-significant difference between genotype groups

46. Literature review

47. Literature review Literature search (criteria) : PubMed January 1- December 31, 2008 Annals of the Rheumatic Diseases & Arthritis and Rheumatism Patients with arthritis were followed up in time over > 3 measurements Response that could vary over time Information collected: Research aim Primary outcome, secondary outcomes Used technique for handling with missing data Statistical analysis techniques used Eligible studies wererankedaccording todegreetheycorrectlyanalyzed the longitudinal research question 47

48. Literature review Study characteristics 203 longitudinal studies in ARD or in AR 156 out of these studies dealt with arthritis 11 studies with JIA patients 58 studies described a RCT 98 studies described a longitudinal cohort study 110studies were included (excluded: 30 studies not longitudinal, 1 study was not first published in 2008, 16 studies did not include patients with arthritis) Statistical techniques used 110 studies were ranked 17 made optimal use of modern statistical methods 48

49. Literature review

50. Conclusions