One-Way Analysis of Covariance (ANCOVA)

One-Way Analysis of Covariance (ANCOVA) • Extension of Analysis of Variance (ANOVA) • One categorical independent (grouping) variable • One continuous dependent variable • Add additional continuous covariate(s) • Covariates hypothesized to have potential effect on outcome of interest • ANCOVA allows statistical adjustment in group analysis, increases likelihood that can detect differences between groups

Uses of ANCOVA • Used when have two-group pre/post-test design (comparing impact on two different interventions, taking before and after measures for each group) • Research Question: Do males and females differ in their reading abilities (measured by reading post-test), following an intervention, controlling for their initial differences in reading (measured by reading pre-test)?

Uses for ANCOVA • Control for pre-existing differences between groups • Control for variables that vary by group and also affect dependent variable (PCV – potentially confounding variables) • Have small samples sizes • Small to medium effect sizes • Quasi-experimental studies where cannot randomly assign study participants to groups

Choosing Covariates • Based on theory and previous research literature guiding your research • Ideally choose 2-3 covariates to reduce error variance and to increase chance of detecting significant differences between groups • Need to be continuous variables • Correlate significantly with dependent variable • Moderately (not highly) correlated with each other • Covariate measured before treatment/intervention so not affected by treatment

Examples - ANCOVA • Is there a significant difference in the Fear of Statistics test scores (FOST) for participants in the math skills group and the confident building group, while controlling for their scores on this test at Time 1? • Is there a difference in self-efficacy levels for low/medium/high performing students, controlling for their parents’ level of education (number of years of formal education completed)?

Assumptions of ANCOVA • Normality • Homogeneity of variances • Influence of treatment on covariate measurement • Reliability of covariates • Multicollinearity • Linearity • Homogeneity of regression • Unequal sample sizes (unbalanced design) • Outliers

Breathe…. • Take deep breaths • Inhale slowly • Hold for 5 seconds • Exhale slowly • Repeat many times

Assumptions Influence of treatment on covariate measurement Ensure covariate measured before the treatment or intervention If violated, covariate may be correlated with dependent variable, thus removing some of treatment effect

Assumptions (cont.) • Reliability of covariates • ANCOVA assumes covariates measured without error (hard to attain) • To minimize violation, need to improve reliability of measurement instruments • Use good, well-validated scales & questionnaires (make sure they measure what you think they measure and are suited for your sample) • Check internal consistency (form of reliability) – Cronbach’s alpha > .7 (a > .8 preferred)

Assumptions (cont.) • Reliability of covariates (cont.) • To minimize violation, need to improve reliability of measurement instruments • If design own instruments, make sure questions clear, appropriate, unambiguous. Pilot-test questions before official data collection! • If using equipment/measuring instrumentation, makes sure it is functioning properly, is calibrated, and that person operating equipment is trained and competent to use. • If study involves other people to observe/rate behavior, make sure they are trained and calibrated to use same criteria. Preliminary pilot-testing to check inter-rater consistency (reliability) is essential.

Assumptions (cont.) • Multicollinearity (a.k.a. correlations among covariates) • To minimize violation, avoid covariates that are highly correlated (strongly related) (r= .8 or above) • Examine scatter plots, run preliminary correlation analyses to examine strength of relationship among proposed covariates • Linearity (a.k.a. linear relationship between dependent variable and covariate) • Use scatter plots to check linearity by subgroup • If curvilinear, eliminate covariate or transform

Add scatter plot example to demonstrate correlation and linearity

Assumptions (cont.) • Homogeneity of regression slopes • Equal “slopes” between covariate and dependent variable • Interaction between covariate and dependent variable is problematic • Unequal sample sizes (unbalanced design) • Outliers • Check on case-by-case basis

Procedures • Analyze General Linear Model, then Univariate Enter Dependent variables, Independent/grouping variable (Fixed factor), covariates Click on Model, Specify Full Factorial Options: Estimated Marginal Means  grouping variable Move into ‘Display Means for’ Options – descriptives, effect size, homogeneity Click OK

An example • Dataset: experim3ED.sav (Pallant) • Use ANCOVA to assess whether there are significant differences between students’ fear of statistics (FOST) following the math skills class (Group 1) or the confidence building class (Group 2), while controlling for their pre-test. • The grouping variable will be: Time. • Check data (Ns for each group, missing data? Coding?) • Check assumptions (e.g., equal variances, linearity)

Experim3ED example (cont.) • Determine overall significance (p<.05) • Compare adjusted means– which is higher? T1 or T2? • Calculate effect size • Present results

Your Turn! • Based on your research interests, what research questions would require an ANCOVA analysis? • Try it out with Omnibus dataset

One-Way Analysis of Covariance (ANCOVA)