1 / 19

Repeated-measures data in educational research trials – how should it be analysed? Ben Styles

Repeated-measures data in educational research trials – how should it be analysed? Ben Styles Senior Statistician National Foundation for Educational Research. Two sweeps example. Cluster randomised trial of reading materials

palma
Download Presentation

Repeated-measures data in educational research trials – how should it be analysed? Ben Styles

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Repeated-measures data in educational research trials – how should it be analysed? Ben Styles Senior Statistician National Foundation for Educational Research

  2. Two sweeps example • Cluster randomised trial of reading materials • Baseline reading test, 10 week intervention, follow-up reading test • Two parallel versions of the Suffolk Reading Scale

  3. Different analysis, different results Using baseline data as a covariate in a multi-level (pupil, school) regression model

  4. Different analysis, different results Using time as a level in a repeated measures multi-level (time, pupil, school) regression model

  5. Interaction

  6. Six sweeps example • Mentoring scheme for struggling readers • Pupil-level randomisation • Questionnaire administered once at baseline and then every four months for the next two years

  7. Six sweeps example Using time as a level in a repeated-measures multi-level (time, pupil, school) regression model

  8. Reading • Two-waves studies cannot describe individual trajectories of change and they confound true change with measurement error (Singer and Willett, 2002) • ANCOVA is valid even with pre-test measurement error (Senn, 2004) • Unconditional change models described in text books have three or more time-points • The ANCOVA will almost always provide a more powerful test of the hypothesis of interest than will the repeated measures ANOVA approach (Dugard and Todman, 1995)

  9. Change model assumption violation (2 sweeps)

  10. Change model assumption OK (six sweeps)

  11. Measurement error problematic

  12. Measurement error problematic

  13. Measurement error not a problem

  14. A better repeated measures model

  15. A (slightly) better conditional model

  16. Conclusion • No consensus but it is probably safer to use a conditional model for a pre-test post-test design • Designs with three or more sweeps will benefit from a repeated measures multi-level model • Care with level 1 residual autocorrelation • Try a few models and check assumptions • Don’t get hung up on significance

  17. Questions and advice

  18. Acknowlegements Pearson Business in the Community and Queen’s University, Belfast Tom Benton Dougal Hutchison NFER

More Related