1 / 39

Issues with Mixed Models

Issues with Mixed Models. Model doesn’t converge…. OR. Convergence. Likelihood Landscape. Likelihood Landscape. Maximum Likelihood Estimation. Mean. Maximum Likelihood Estimation. Mean. Maximum Likelihood Estimation. Mean. Maximum Likelihood Estimation. Mean.

dyanne
Download Presentation

Issues with Mixed Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Issues withMixed Models

  2. Model doesn’t converge… OR

  3. Convergence

  4. Likelihood Landscape

  5. Likelihood Landscape

  6. Maximum Likelihood Estimation Mean

  7. Maximum Likelihood Estimation Mean

  8. Maximum Likelihood Estimation Mean

  9. Maximum Likelihood Estimation Mean

  10. Maximum Likelihood Estimation Likelihood = the probability of seeing the data we actually collected given a particular model Maximum Likelihood Estimates = those values that make the observed data most likely to have happened

  11. Sources of Convergence Problems • You estimate more parameters than data (or, in general, too many parameters • Severe collinearity (e.g., two predictors are exactly correlated) • Missing cells in your design • Predictors of vastly different metrics

  12. Failure to converge GENDER ATTITUDE male female polite 16 0 informal 16 32 … and then trying to test the ATTITUDE*GENDER interaction

  13. How can this happen? “Death by Design” (coined byRoger Mundry)

  14. design analysis

  15. Solutions to Convergence Problems • Drop a random slope(not preferred, should be reported) • Drop subjects/items for which there is not enough data (not preferred, should be reported) • Rescale variables so that they lie range between 0 and 1; or make them on similar metrics overall • Center continuous predictors • Nonlinear transformations of skewed predictors

  16. Solutions to Convergence Problems • Change order of variable names in model formula • Have a balanced and complete design

  17. p-values

  18. The p-value conundrum What are the degrees of freedom? Howto get p-values out ofmixedmodels is not entirely straightforward… DouglasBates

  19. “There are a number of ways to compute p-values from LMEMs, none of which is uncontroversially the best.” Barr et al. (2013)

  20. Ways to get p-values • t-test/F-test with normal approximation • Likelihood Ratio Test • Boostrapping • Permutation • Markov Chain Monte Carlo (MCMC)

  21. Getting p-vals with normal approximation xmdl coefs=data.frame(summary(xmdl)@coefs) coefs$p = 2*(1-pnorm(abs(coefs$t.value))) coefs

  22. Function for getting p-vals with normal approximation create.sig.table = function(x){ coefs=data.frame(summary(x)@coefs) coefs$p = 2*(1-pnorm(abs(coefs$t.value))) coefs$sig = character(nrow(coefs)) coefs[which(coefs$p < 0.05),]$sig = "*" coefs[which(coefs$p < 0.01),]$sig = "**" coefs[which(coefs$p < 0.001),]$sig = "***" return(coefs) }

  23. Likelihood Ratio Test First model needs to be nested in second

  24. Likelihood Ratio The likelihood ratio expresses how many times more likely the data are under one model than the other Mean Mean

  25. Likelihood Ratio Test

  26. Likelihood Ratio Test

  27. Important whendoing likelihood ratio tests lmer(…,REML=FALSE) http://anythingbutrbitrary.blogspot.com/2012/06/random-regression-coefficients-using.html

  28. Final issue:Random slopes

  29. Random intercept onlymodels are known to beveryanti-conservativein many circumstances DANGEROUS!!! (cf. Barr et al., 2013,Schielzeth & Forstmeier, 2008)

  30. Random intercept only Schielzeth & Forstmeier(2008)

  31. Type I error simulation 10 subjects 10 data points each 5 of those in condition A,5 in B

  32. Add to this explicit subject slopesfor A/B 10 subjects 10 data points each 5 of those in condition A,5 in B

  33. Add to this explicit subject slopesfor A/B 10 subjects 10 data points each 5 of those in condition A,5 in B

  34. Add to this explicit subject slopesfor A/B + take item slopes 10 subjects 10 data points each 5 of those in condition A,5 in B

  35. Add to this explicit subject slopesfor A/B + take item slopes 10 subjects 10 data points each 5 of those in condition A,5 in B

  36. “Keep it maximal”

  37. “Keep it maximal” random effectsjustified by the design vs. random effectsjustified by the data Barr et al. (2013)

  38. “Keep it maximal” “for whatever fixed effects are of critical interest, the corresponding random effects should be in that analysis” Barr et al. (2013)

  39. That’s it(for now)

More Related