1 / 43

Analysis of Repeated Measures Will G Hopkins, Auckland University of Technology, Auckland, NZ

Analysis of Repeated Measures Will G Hopkins, Auckland University of Technology, Auckland, NZ. A tutorial lecture presented at the 2003 annual meeting of the American College of Sports Medicine.

jaden
Download Presentation

Analysis of Repeated Measures Will G Hopkins, Auckland University of Technology, Auckland, NZ

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis of RepeatedMeasuresWill G Hopkins, Auckland University of Technology, Auckland, NZ A tutorial lecture presented at the 2003 annual meetingof the American College of Sports Medicine • This presentation applies to continuous or ordinal numeric dependent variables, including data from most Likert scales. • It does not apply to nominal dependent variables or variables representing counts or frequencies. • Make sure you view this presentation as a full slide show, to get the benefit of the build-up of information on each slide.

  2. OVERVIEW Basics • What change has occurred in response to a treatment/intervention? • Analysis by ANOVA, within-subject modeling, mixed modeling. • Fixed and random effects; individual responses and asphericity. Accounting for Individual Responses • What is the effect of subject characteristics on the change? Analyzing for Patterns of Responses • What is the treatment's effect on trends in repeated sets of trials? Analyzing for Mechanisms • How much of the change was due to a change in whatever?

  3. Basics What change has occurred in response to a treatment or intervention?

  4. Period oftreatment exptal Data are means and standard deviations pre mid post control Trial or Time • Within-subjects factor • Same subjects on each level Group • Between-subjects factor • Different subjects on each level Basics: Interventions • A repeated measure is a variable measured two or more times, usually before, during and/or after an intervention or treatment. Y • Dependent variable • Repeated measure • Analysis by ANOVA, t statistics and within-subject modeling, and mixed modeling.

  5. Measure = "Y"within-subjects factor = "Trial" Girl Group Ypre Ymid Ypost Ann exptal 58 62 68 Bev exptal 45 . 57 Lyn control 39 42 40 May control 44 45 42 Missing value means loss of subject. Basics: Analysis by ANOVA • Data are in the form ofone row per subject: • Select columns to definea within-subjects factor. • If there is no control group, use a 1-way repeated-measures ANOVA • The 1 way is Trial: • "(How) does Trial affect Y?" • With a control group, use a 2-way repeated-measures ANOVA. • The 2 ways are Group and Trial. • You investigate the interaction GroupTrial: • "(How) does Trial affect Y differently in the different groups?"

  6. Ypost-Ypre Girl Group Ypre Ymid Ypost 10 Ann exptal 58 62 68 Bev exptal 45 . 57 12 Lyn control 39 42 40 1 Missing value does not affect post – pre changes May control 44 45 42 -2 Basics: Analysis by t Statistics and Within-Subject Modeling • If there is no control group, use a paired t statistic to investigate changes between interesting measurements. • With a control group, calculate change scores and use the unpaired t statistic to investigate the difference in the changes. • Use un/paired t statistics for other interesting combinations of repeated measurements. I call it within-subject modeling. • Example: time course of an effect…

  7. Missing valueno problem. Basics: More Within-Subject Modeling Ann • To quantify a time course: • fit lines or curves to each subject's points; • predict interesting things for each subject; • analyze with un/paired t statistic. • Method #1. Fit lines Y= a + b.T • At Time 0 and 3, Y = a and a+3b. • Change in Y = b per week. • Method #2. Fit quadratics Y= a + b.T + c.T2 • At Time 0 and 3, Y = a and a+3b+9c. • Change in Y = 3b+9c over 3 weeks. • Maximum occurs at Time = -b/(2a). • Method #3. Fit exponentials Y= a + b.eT/c • Needs non-linear curve fitting to estimatetime constant c. Bev Lyn Y May 3 1 2 0 Time (wk)

  8. Girl Group Trial Y Ann exptal pre 58 Ann exptal mid 62 Ann exptal post 68 Bev exptal pre 45 Bev exptal mid . Bev exptal post 57 Lyn control pre 39 Lyn control mid 42 Missing value means loss of only one trial for the subject. Basics: Analysis by Mixed Modeling • Data are in the form of one row per subject per trial: • Analysis is via maximizing likelihood of observed valuesrather than ANOVA's approachof minimizing error variance. • You investigate fixed effects: • Trial, if there's only one group. • GroupTrial, if there's morethan one group. • You also specify and estimate random effects. • "Mixed" = fixed + random. • Some mixed models are also known as hierarchical models.

  9. Basics: Fixed Effects • Fixed effects are differences or changes in the dependent variable that you attribute to a predictor (independent) variable. • They are usually the focus of our research. • Their value is the same (fixed) for everyone in a group. • They have magnitudes represented by differences or changesin means. • Example of difference in means: • girls' performance = 48 • boys' performance = 56 • so effect of sex (maleness) on performance = 56 – 48 = 8. • Example of change in a mean: • girls' performance in pretest = 48 • girls' performance after a steroid = 56 • so effect of the steroid on girls' performance = 56 – 48 = 8.

  10. Basics: Random Effects • Random effects have values that vary randomly within and/or between individuals. • They provide confidence limits or p values for the fixed effects. • They provide other valuable information usually overlooked. • They are mostly hidden in ANOVA, are accessible in t tests, and are up front in mixed modeling. • They are the key to understanding repeated measures. • They have magnitudes represented by standard deviations (SD). • Examples of between-subject SD or random effects: • Variation in ability: SD of girls' performance (Y) = 9.2 • Individual responses: SD of effect of a steroid on Y = 5.0, so you can say the effect of the steroid is 8.0 ± 5.0 (mean ± SD). • Example of a within-subject SD or random effect: • Error of measurement: SD of any girl's Y in repeated tests = 2.0

  11. A girl's true performance (not observed) A girl's observed performance… in Trial #1 in Trial #2 48.3 + +7.4 =55.7 55.7 + +2.1 =57.8 55.7 + -1.3 =54.4 SD = 9.2 SD = 2.0 Girls Trials Basics: The "Hats" Metaphor for Random Effects • When you measure something, it's like adding together numbers drawn from several hats. • Each hat holds a zillion pieces of paper, each with a number. • The numbers are normally distributed with mean = 0, SD = ?? • Example: measure a girl's performance several times. Suppose true mean performance of all girls = 48.3 • The random effects in SAS are Girl and GirlTrial (= the residuals).

  12. Performance in Trial #1 Performance in Trial #2 + Ann 55.7 + +2.1 =57.8 -1.3 55.7 + =62.4 8.0 + Bev 48.4 + -3.1 =45.3 48.4 + +0.7 =57.1 8.0 SD = 2.0 Trials + Cas 65.2 + -2.8 =62.4 65.2 + -1.4 =71.8 8.0 + Deb 40.7 + +0.5 =41.2 40.7 + +2.8 =51.5 8.0 SD = 2.0 Trials SD = 2.0 Trials • These are all we can observe. SD = 2.0 Trials Basics: Hats plus a Fixed Effect • Example: give steroid with a fixed effect of 8.0 between Trials #1 and #2, and measure several girls. • Subject hat not shown. • The stats program uses them to estimate the fixed and random effects.

  13. Performance in Trial #1 Performance in Trial #2 + +5.2 + Ann 55.7 + +2.1 =57.8 55.7 + -1.3 =67.6 8.0 + -0.5 + Bev 48.4 + -3.1 =45.3 48.4 + +0.7 =56.6 8.0 + +6.2 + Cas 65.2 + -2.8 =62.4 65.2 + -1.4 =78.0 8.0 SD = 2.0 SD = 5.0 Individ Trials Responses + -2.7 + Deb 40.7 + +0.5 =41.2 40.7 + +2.8 =48.8 8.0 SD = 2.0 SD = 5.0 Individ Trials Responses SD = 2.0 SD = 5.0 Individ Trials Responses SD = 2.0 SD = 5.0 Individ Trials Responses Basics: A Hat for Individual Responses • Example: different responses to the steroid. • To estimate the SD for individual responses, you need a control group (see later) or an extra trial for the treatment group.

  14. Basics: Individual Responses and Asphericity • It's important to quantify individual responses, but… • More importantly, they are the most frequent reason for the asphericity type of non-uniform error in repeated measures. • You must somehow eliminate non-uniformity of error to get trustworthy confidence limits or p values. • Here's the deal on asphericity. • Conventional ANOVA is based on the assumption that there is only one random-effects hat, error of measurement. • We can use ANOVA for repeated measures by turning the subjects random effect into a subjects fixed effect. • But it doesn't work properly when there is asphericity: that is,more than one source of error, such as individual responses. • There are four approaches to the asphericity problem.

  15. Basics: Dealing with Asphericity in Repeated Measures • Four approaches: • MANOVA (multivariate ANOVA) • (Univariate) ANOVA with adjustment for asphericity • Within-subject modeling with the unequal-variances t statistic • Mixed modeling • I base my assessment of these approaches mainly on my experience with the Statistical Analysis System (SAS). • Other stats programs may produce different output.

  16. Basics: MANOVA/adjusted ANOVA for Asphericity (NOT!) • Both these approaches involve different assumptions about the relationship between the repeated measurements. • They produce an overall p value for each fixed effect. • Incredibly, the p value is too small if sample size and individual responses differ between groups. • Adjusted ANOVA (Greenhouse-Geisser or Huynh-Feldt) is worse than MANOVA. • Subjects with any missing value are first deleted. • So there is needless loss of power, if the missing value is for a minor repeated measurement (e.g., post2). • In the old-fashioned approach, you are allowed to "test for where the difference is" only if the overall p<0.05. • So there is further loss of power, because you could fail to detect an effect on the overall p or the subsequent test.

  17. Basics: More on MANOVA/adjusted ANOVA • The overall p value is OK when the extra random effects are the same in both groups, even when sample sizes differ. • Example: two repeated-measures factors; for example, several measurements on one day repeated at monthly intervals. • The program then does p values for the requested contrasts (differences in the changes; e.g., post – pre for exptal – control). • These comparisons are simply equal-variance t tests. • So the p values are too small if sample size and individual responses differ between groups. • There is no adjustment other than Bonferroni for inflation of Type I error for contrasts involving repeated measures. • Good! But researchers still dial up Tukey or other adjustments and think that the resulting p values are adjusted. They're not. • In summary: avoid MANOVA and adjusted ANOVA.

  18. + + = 33 Y exptal Trials Trials Trials Trials + = 8 control SD2 = 4 SD2 = 4 SD2 = 4 SD2 = 4 pre post Randomeffects: Individ Responses Individ SD2 = 25 Trials Responses SD = 2.0 SD = 5.0 Basics: Unequal-Variances t Statistic Deals with Asphericity • Example: controlled trial of effect of the steroid on performance. Variance of post–pre change scores: • Big differences in variances. • So use unequal-variances t statistic to analyze changes. • Bonus: estimate of individual responses as an SD =(SDChgExpt2– SDChgCont2)

  19. Basics: Summary of t Statistic for Repeated Measures • Advantages • It works! • It's robust to gross departures from non-normality, provided sample size is reasonable. • 10 in each group is forgiving, 20 is very forgiving. • Missing values are not a problem. • Because you analyze separately the changes of interest. • Students can do most analyses with Excel spreadsheets. • Include my spreadsheet for confidence limits and clinical/practical/mechanistic probabilities. • You can include covariates by moving to simple ANOVAs or ANCOVAs of the change scores. • Example: how does age modify the effect of the steroid on performance? (See later.) But…

  20. Basics: More on t Statistic for Repeated Measures • Disadvantages • ANOVAs or ANCOVAs of the change scores aren't strictly applicable, if variances of the change scores differ markedly. • You can't easily get confidence limits for the SD representing individual responses. • That is, I don't have a formula or spreadsheet yet. • There's always bootstrapping, but it's hard work. • The disdain of editors and peer reviewers, most of whom think state of the art is repeated-measures ANOVA with post-hoc tests controlled for inflation of Type I error. • In conclusion, I recommend within-subject modeling using unequal-variances t statistic for analysis of straightforward data. • Otherwise use mixed modeling…

  21. Basics: Mixed Modeling for Asphericity • You take account of potential sources of asphericity by including them as random effects. • Advantages • It works! • Impresses editors and peer reviewers. • Confidence limits for everything. • Complex fixed-effects models are relatively easy: • individual responses, patterns of responses, mechanisms • Disadvantages • Not available in all stats programs. • Takes time and effort to understand and use. • The documentation is usually impenetrable. • Sample size for robustness to non-normality not yet known.

  22. Accounting forIndividual Responses What is the effect of subject characteristics on the change?

  23. boys girls Y Data arevalues forindividuals pre mid post pre mid post Trial Individual Responses: and Subject Characteristics • Subjects differ in their response to a treatment… …due to subject characteristics interacting with the treatment. • It's important to measure and analyze their effect on the treatment. • Using value of Trialpre as a characteristic needs special approach to avoid artifactual regression to the mean. See newstats.org. • Use mixed modeling, ANOVA, or within-subject modeling.

  24. Individual Responses: byMixed Modeling • You include subject characteristics as covariates in the fixed-effects model. • The SD representing individual responses will diminish and represent individual responses not accounted for by the covariate. • The precision of the estimates of the fixed effects usually improves, because you are accounting for otherwise random error. • Covariates can be nominal (e.g., sex) or numeric (e.g., age). • Example: how does sex affect the outcome? • First, you can avoid covariates by analyzing the sexes separately. • Effect on females = 8.8 units; effect on males = 4.7 units. • Effect on females – males = 8.8 – 4.7 = 4.1 units. • You can generate confidence limits for the 4.1 "manually", by combining confidence limits of the effect for each sex. • Include individual responses for each sex: 8.8 ± 5.2; 4.7 ± 2.5.

  25. Individual Responses: More Mixed Modeling • The full fixed-effects model is Y GroupTrial SexGroupTrial. • The term SexGroupTrial yields the female-male difference of 4.1 units (90% confidence limits 1.5 to 6.7, say). • The overall effect of the treatment (from GroupTrial) is for an average of equal numbers of females and males. • Try including random effects for individual responses in males and females. • Example: how does age affect the outcome? • Either: convert age into age groups and analyze like sex. • Or: if the effect of age is linear, use it as a numeric covariate. • AgeGroupTrial provides the outcome as effect per year: 1.3 units.y-1 (90% confidence limits -0.2 to 2.8). • Note that the overall effect of the treatment is for subjects with the average age.

  26. Individual Responses: by Repeated-Measures ANOVA • It is possible in principle to include a subject characteristic as a covariate in a repeated-measures ANOVA. • But SPSS (Version 10) provides only the p value for the interaction. Incredibly, it does not provide magnitudes of the effect. • If a covariate accounts for some or all of the individual responses, the problem of asphericity will diminish or disappear. • I don't know whether it's possible to extract the SD representing individual responses from a repeated-measures ANOVA, with or without a covariate.

  27. Ypost-Ypre Kid Sex Age Group Ypre Ymid Ypost Ann F 23 exptal 58 62 68 10 4 Ben M 19 exptal 64 67 68 Lyn F 19 control 39 42 40 1 Merv M 19 control 59 60 57 -1 Individual Responses: byWithin-Subject Modeling • Calculate the most interesting change scores or other within-subject parameters: • If no control group, analyze effect of subject characteristics on change score with unpaired t, regression, or 1-way ANOVA. • With a control group, analyze with 2-way ANOVA. • As before, a characteristic that accounts partially for individual responses will reduce the problem of asphericity.

  28. Analyzing for Patterns of Responses What is the effect of a treatment on trends within repeated sets of trials?

  29. 1 2 Bout 3 exptal 4 Y Standard deviations: Between Subjects within Bout control Within Subject between Trials pre mid post Within Subject within Trial Trial Patterns of Responses: Bouts within Trials • Typical example: several bouts for each of several trials. • We want to estimate the overall increase in Y in the exptal group in the mid and post trials, and… • …the greater decline in Y in the exptal group within the mid and post trials (representing, for example, increased fatigue). • Use mixed modeling, ANOVA, or within-subject modeling.

  30. Patterns of Responses: byMixed Modeling and ANOVA • With mixed modeling, Bout is simply another (within-subject) fixed effect you add to the model. • The model is Y  Trial Bout TrialBout. • Bout can be nominal or numeric. • If numeric, Bout specifies the slope of a line, and TrialBout specifies a different slope for each level of Trial. • Add BoutBout(Trial) to the model for quadratic(s). • Elegant and easy, when you know how. • With ANOVA, you have to specify Bout as a nominal effect and try to take into account within-subject errors using adjustments for asphericity. • Specifying a quadratic or higher-order polynomial Bout effect is possible but difficult (for me, anyway). • Within-subject modeling is much easier…

  31. Subject: JC Y pre mid post Trial Patterns of Responses: byWithin-Subject Modeling • The trick is to convert the multiple Bout measurements into a single value for each subject, then analyze those values. • In the example, derive the Bout mean and slope(or any other parameters) within each trial for each subject. • Derive the change in meanand the change in slopebetween pre and post(or any other Trials) for each subject. • For the changes in the mean, do an unpaired t test between the exptal and control groups. Ditto for the changes in the slope. • Simple, robust, highly recommended!

  32. Analyzing for Mechanisms How much of the change was due to a change in whatever?

  33. Mechanism variable Dependent variable exptal exptal control control pre mid post pre mid post Trial Analyzing for Mechanisms • Mechanism variable = something in the causal path between the treatment and the dependent variable. • Necessary but not sufficient that it "tracks" the dependent. • Important for PhD projects or to publish in high-impact journals. • It can put limits on a placebo effect, if it's not placebo affected. • Can't use ANOVA; can use graphs and mixed modeling.

  34. Measure = "Y"within-subjects factor = "Trial" Mechanism variable(within-subjects covariate) Girl Group Ypre Ymid Ypost Xpre Xmid Xpost Ann exptal 58 62 68 8.4 8.7 9.1 Bev exptal 45 . 57 9.0 . 9.7 Lyn control 39 42 40 7.9 7.7 7.8 May control 44 45 42 7.1 7.1 7.2 Mechanisms: Why not ANOVA? • For ANOVA, data have to be one row per subject: • You can't use ANOVA, because it doesn't allow you to match up trials for the dependent and covariate.

  35. Change scorefor dependent Ypost-Ypre Xpost-Xpre Girl Group Ypre Ymid Ypost Xpre Xmid Xpost Ann exptal 58 62 68 8.4 8.7 9.1 10 1.5 Bev exptal 45 . 57 9.0 . 9.7 12 0.7 Lyn control 39 42 40 7.9 7.7 7.8 1 -0.1 May control 44 45 42 7.1 7.1 7.2 -2 0.1 Change scorefor covariate Mechanisms: Analysis Using Graphs • Choose the most interesting change scoresfor the dependent and covariate: • Then plot the change scores…

  36. 1. Large individual responses… …tracked by mechanism variable… …even in the control group. exptal Ypost - Ypre 0 control 0 Xpost - Xpre Mechanisms: MoreAnalysis Using Graphs • Three possible outcomes with a real mechanism variable: • The covariate is an excellent candidate for a mechanism variable.

  37. Ypost - Ypre 0 0 Xpost - Xpre Mechanisms: MoreAnalysis Using Graphs • Three possible outcomes with a real mechanism variable: 2. Apparently poor tracking of individual responses… … but it could be due to noise in either variable. • The covariate could still be a mechanism variable.

  38. 3. Little or no individual responses… …but mechanism variable tracks mean response. Ypost - Ypre 0 0 Xpost - Xpre Mechanisms: MoreAnalysis Using Graphs • Three possible outcomes with a real mechanism variable: • The covariate is a good candidate for a mechanism variable.

  39. Ypost – Ypre 0 0 0 Xpost – Xpre Mechanisms: Graphical Analysis – how NOT to • Relationship between change scores is often misinterpreted.  • "The correlation between change scores for X and Y is trivial. • Therefore X is not the mechanism."   • "Overall, changes in X track changes in Y well, but… • Noise may have obscured tracking of any individual responses. • Therefore X could be a mechanism variable." 

  40. Mechanism variable(within-subjects covariate) Girl Group Trial Y X Ann exptal pre 58 8.4 Ann exptal mid 62 8.7 9.1 Ann exptal post 68 Bev exptal pre 39 9.0 • No problem with aligning trials for the dependent and covariate. Mechanisms: Quantitative Analysis by Mixed Modeling - 1 • Need to quantify the role of the mechanism variable, with confidence limits. • I have devised a method using mixed modeling. • Data format isone row per trial:

  41. Mechanisms: MoreQuantitative Analysis by Mixed Modeling • Run the usual fixed-effects model to get the effect of the treatment. • Example: 4.6 units (90% likely limits, 2.1 to 7.1 units). • Then include a putative mechanism variable in the model. • The model is then effectively a multiple linear regression, so… • You get the effect of the treatment with the mechanism variable held constant… • …which means the same as theeffect of the treatment not explained by the putative mechanism variable. • Example: it drops to 2.5 units (90% likely limits, -1.0 to 7.0 units). • So the mechanism accounts for 4.6 - 2.5 = 2.1 units. • If the experiment was not blind, the real effect is >2.1 units… • …and the placebo effect is <2.5 units... • …provided the mechanism variable itself is not placebo affectible!

  42. Summary Basics • Use the unequal-variance t statistic and within-subject modeling for straightforward models. • Repeated-measures ANOVA may not cope with non-uniform error. • Mixed modeling is best for fixed and random effects. Accounting for Individual Responses • Use within-subject modeling or mixed modeling. Analyzing for Patterns of Responses • Use within-subject modeling or mixed modeling. Analyzing for Mechanisms • Interpret graphs of change scores properly. • Use mixed modeling to get estimates of the contribution of a mechanism variable.

  43. This presentation was downloaded from: A New View of Statistics newstats.org SUMMARIZING DATA GENERALIZING TO A POPULATION Simple & Effect Statistics Precision of Measurement Confidence Limits Statistical Models Dimension Reduction Sample-Size Estimation

More Related