Understanding Three-Level Models In Inference | Analysis of Variance Components and Correlations

Stat 414 – Day 17 Three-level models Inference

Last Time – 3-level models • Yijk = grand mean + random effect for school + random effect for class + random effect for student • 3 variance components • Proportion of total variation at each level (VPC) • Intraclass correlation coefficients (ICC) • Level 3/total: correlation of students in same school • (Level 2 + Level 3)/total: correlation of students in same class (in same school) • Level 3/(Level 2 + Level 3): correlation of classes within same school • What is correlation of student with self? • What is correlation of 2 students from different schools?

Last Time – 3-level models

class Covariances… • Divide by total variance to get correlations 1 2 3 4 5 6 1 2 3 4 5 6

Day 16 Example 3 (achieve.txt) • JMP SDs .5584 .5222 2.2016

Example 3 (achieve.txt) • JMP R - lme by REML AIC BIC logLik 46154 46182.97 -23073 SDs .5584 .5222 2.2016

Example 3 (achieve.txt) • JMP R - lme (method = “ML”) by ML AIC BIC logLik 46150.04 46179 -23071.02 SDs .5584 .5222 2.2016

Example 3 (achieve.txt) • JMP R - lmer SDs .5584 .5222 2.2016

Example 3 (achieve.txt) • JMP R – lmer (REML= F) SDs .5584 .5222 2.2016

How compare models? • Model with no random effects? • Model with only Level 2 random effects? • Model with only Level 3 random effects?

Possible approaches • R, JMP lme: intervals(model1) • JMP: wald tests • These can be risky with small sample sizes as assume normality and variance estimates follow skewed distributions • Likelihood ratio tests • p-value actually a bit conservative • R lmerTest: likelihoods change • ML (log likelihood) vs. REML (residual log like)

Opinions vary • Use ML when dropping fixed effects terms • You can compare nested models that only differ in the random terms by using the REML likelihood or the ordinary likelihood. If you want to compare models that differ in fixed effects terms (or both), then use ordinary likelihood.

Example With lme (lmer will do either) 46758.087 – 46142.04 = 616.05

Adding predictors • Let’s add student’s vocabulary (genvocab), number of students in class (clenroll), number of students in school (cenroll)

Understanding Three-Level Models In Inference | Analysis of Variance Components and Correlations