Introduction the General Linear Model (GLM)

1. Introduction the General Linear Model (GLM) what “model,” “linear” & “general” mean bivariate, univariate & multivariate GLModels kinds of variables some common models

2. “General Linear Model” • “Model” means that we are usually interested in predicting or “modeling” the values of one variable (criteria) from the values of one or more others (the predictors) • “Linear” means that the variables will be “linearly transformed (* & /) and “linearly combined” (+ & -) to produce the model’s estimates • “General” means that the model intends to provide a way to model & test RHs: about any combination of criterion and predictor variables (i.e., any model), and to test RHs: about comparisons among models

3. Regression “vs.” GLM The “constant” is often represented differently in GLM than in multiple regression … Single predictor models  single predictor regression y’ = bx + a  single predictor GLM y’ = b0 + b1x1 Multiple predictor models • multiple predictor regression y’ = b1x1 b2x2+ a • multiple predictor GLM y’= b0 + b1x1 + b2x2

4. Common kinds of GLModels • Bivariate one criterion & one predictor simple regression y’ = b0 + b1x • Univariate  one criterion & multiple predictors  multiple regression in all its forms y’ = b0 + b1x1 + b2x2 + b3x3 • Multivariate  multiple criterion & multiple predictors canonical regression in all its forms b0 + b1y1 + b2y2= b0 + b1x1 + b2x2 + b3x3

5. Common kinds of variables Quantitative variables • Raw variable • Centered variables X – mean • Mean  0 simplifies math of more complicated models • Re-centered variables X  a more meaningful value • Change “start” or “stop” values • E.g., “aging & intellectual decline” • Mathematical “trick” to get the desired model/weights • selecting which group or value will be represented in model’s bs

6. Common kinds of variables Quadratic quantitative variables • X2 – actually represents combination of linear + quadratic • Xcen2 – represents the “pure” quadratic term • Model with X2 will have ≈R2 as model with Xcen + Xcen2 • A model with a quadratic term should always include the linear term for that variable

7. Common kinds of variables 2-group variables • Unit coding (usually 1-2) • Dummy Coding • “control” or “comparison” group coded 0 • “treatment” or “target” group coded 1 • Effect Coding • “control” or “comparison” group coded -1 • “treatment” or “target” group coded 1

8. Common kinds of variables k-group variables • Raw coding (usually 1-2-3, etc.) • Dummy Coding • “control” or “comparison” group coded 0 • “treatment” or “target” groups coded 1 on one variable & 0 on all others • the full set of codes must be included in the model • Effect Coding • “control” or “comparison” group coded -1 • “treatment” or “target” group coded 1 on one variable & 0 on all others • the full set of codes must be included in the model

9. Common kinds of variables K-groups variables, cont. • Comparison coding • Combining simple and complex analytical comparison codes to represent specific, hypothesis driven, group comparisons • E.g., Say you have 4 groups and RH: that… • Group 1 has higher scores that the average scores of groups 2-4 the codes would be gp1 = 3 gp2 = -1 gp3 = -1 gp4 = -1 • Groups 2 & 3 have higher average scores than do 1 & 4 the codes would be gp1 = -1 gp2 = 1 gp3 = 1 gp4 = -1 • Group 2 has higher scores than the average scores of groups 3-4 the codes would be gp1 = 0 gp2 = 2 gp3 = -1 gp4 = -1 • Usually havea set of k-1 codes

10. Common kinds of variables K-groups variables, cont. Polynomial coding If the groups represent a quantitative continuum, you use codes to represent different polynomial functions (linear, quadratic, cubic, etc.) to explore the shape of the relationship between that variable and the criterion E.g., for a 5-group variable, the polynomial codes are … • Linear -2 -1 0 1 2 • Quadratic 2 -1 -2 -1 2 • Cubic -1 2 0 -2 1 • Quartic 1 -4 6 -4 1 • the full set of codes must be included in the model

11. Common kinds of variables Ordered-category variables Sometimes you have a quantitative variable that you want to change into a set of ordered categories e.g.  % grade into “A” “B” “C” “D” “F” e.g.  % grade into “Pass” “Fail” e.g.  aptitude test scores into “remedial” “normal” “gifted” Sometimes this is done to help with “ill-behaved distributions” e.g.  frequency variable with mean=1.1, std=8.4, sk=4.2 e.g.  frequency variable with 60% “0” 38% “1” max = 118 Important because  skewed univariate distributions can “create” apparently nonlinear bivariate relationships

12. Common kinds of variables Ordered-category variables, cont. Once you form the ordered categories (using “IF,” “RECODE” or other transformations), you can enter those variables into the GLM in different ways • Using the category values (e.g., 1, 2, 3, etc)*** • Centering or re-centering the category values*** • Dummy codes of the category values • Effect codes of the category values • Polynomial codes of the category values*** *** indicates approaches that make assumptions about the interval nature of the variable and/or its normal distribution, with which not everyone agrees!

13. Common kinds of variables Interactions • Interactions represent the “joint effect” or “non-additive combination” of 2 or more predictors as they relate to a criterion (or set of criteria in the multivariate case). • They are the “moderation,” “it depends,” “sometimes,” or “maybe” that makes our science and statistical analyses so interesting. • Interactions can be formed from the combination of any 2 or more variables of the types just discussed. • There are some “guidelines” about forming, including and interpreting interaction terms.

14. Common kinds of variables Interaction  “Guidelines” • When including a 2-way interaction, both related main effects must be included • When including a 3-way interaction, all 3 main effects and all 3 2-way interactions must be included • When including a non-linear interaction term, the related linear and nonlinear main effects, and linear interaction terms must be included • The associated terms can not exceed the df of the variables involved (except for quantitative variables)

15. Common kinds of GLModels “Linear” Multiple regression models y’ = b0+ b1x1+ b2x2+b3x3 Can include any of the variable types: • Quantitative (raw, centered or re-centered) • 2- or k-group (with dummy, effect, or comparison coding) • Ordered category (coded)

16. Common kinds of GLModels “Non-Linear” Multiple regression models with quant variables y’ = b0+ b1x1+ b2x12+b3x2 + +b4x22 Can include any of the variable types: • Linear terms should be centered • Non-linear terms should be centered then powered • Non-linear terms above quadratic should be based on theory • Include linear term for all non-linear terms, at least at first

17. Common kinds of GLModels “2-way Interaction” Multiple regression models y’ = b0+ b1x+ b2z+ b3xz Can include any of the variable types: • Quantitative variables should be centered • 2- or k-group variables should be coded • Interaction terms formed as product of main effect terms • Must included main effects terms for any interaction variable

18. Common kinds of GLModels “3-way Interaction” Multiple regression models y’ = b0+ b1x+ b2z+ b3v+ b4xz+ b5xv+ b6zv +b7xzv Can include any of the variable types: • Quantitative variables should be centered • 2- or k-group variables should be coded • Interaction terms formed as product of main effect terms • Must included main effects terms for any interaction variable

19. Common kinds of GLModels 2-group ANOVA models y’ = b0+ b1x1 “X” is a dummy or effect coded 2-group variable

20. Common kinds of GLModels 3-group ANOVA models y’ = b0+ b1x1+ b2x2 “X1” & “X2” are a dummy or effect codes for a 3-group variable

21. Common kinds of GLModels 4-group ANOVA models y’ = b0+ b1x1+ b2x2+ b3x3 “X1,” “X2” & “X3” are a dummy or effect codes for a 4-group variable

22. Common kinds of GLModels 2x2 Factorial ANOVA model y’ = b0+ b1x+ b2z+ b3xz “X” is a dummy or effect code of 1st 2-group variable “Z” is a dummy or effect code of 2nd 2-group variable “XZ” represents the interaction of “X” and “Z”

23. Common kinds of GLModels 2x3 Factorial ANOVA model y’ = b0 + b1x1+ b2z1+ b3z2+ b4xz1+ b5xz2 “X1” is a dummy or effect code of 1st 2-group variable “Z1” & “Z2” are dummy or effect codes of 2nd k-group variable “XZ1” & “ZX2” represent the interaction of “X” and “Z”

24. Common kinds of GLModels 2-group ANCOVA models y’ = b0+ b1x+ b2z “X” is a dummy or effect coded 2-group variable “Z” is the covariate (dummy coded or quantitative)

25. Common kinds of GLModels 2-group ANCOVA models with covariate interaction y’ = b0+ b1x+ b2z+ b3xz “X” is a dummy or effect coded 2-group variable “Z” is the covariate (dummy coded or quantitative) “XZ” represents the interaction of “X” and “Z”

26. Common kinds of GLModels 3-group ANCOVA models y’ = b0+ b1x1+ b2x2+ b3z “X1” & “X2” are a dummy or effect codes for a 3-group variable “Z” is the covariate (dummy coded or quantitative)

27. Common kinds of GLModels 3-group ANCOVA model with covariate interaction y’ = b0+ b1x1+ b2x2+ b3z+ b4xz1+ b5xz2 “X1” & “X2” are a dummy or effect codes for a 3-group variable “Z” is the covariate (dummy coded or quantitative) “XZ1” & “ZX2” represent the interaction of “X” and “Z”

28. Common kinds of GLModels 2x2 Factorial ANCOVA model y’ = b0+ b1x+ b2z+ b3xz+ b4v “X” is a dummy or effect code of 1st 2-group variable “Z” is a dummy or effect code of 2nd 2-group variable “XZ” represents the interaction of “X” and “Z” “V” represents the covariate

29. Common kinds of GLModels 2x2 Factorial ANCOVA model with covariate interactions y’ = b0+ b1x+ b2z+ b3xz+ b4v+ b5xv+ b6zv+ b7xzv “X” is a dummy or effect code of 1st 2-group variable “Z” is a dummy or effect code of 2nd 2-group variable “XZ” represents the interaction of “X” and “Z” “V” represents the covariate “XV” represents the interaction of “X” and “V” “ZV” represents the interaction of “Z” and “V” “XZV” represents the interaction of “X,” “Z” and “V”