Education 795 Class Notes

1. Education 795 Class Notes Non-Experimental Designs ANCOVA Note set 5

2. Today’s Agenda • Announcements (ours and yours) • Q/A • Non-experimental design • Categorical predictors • Statistical control • ANCOVA • Interactions

3. Nonexperimental Designs • What distinguishes designs are • a. manipulation of IV’s • b. randomization • Experiments have both a and b • Quasi-experiments have a but not b • Nonexperimental designs have neither a nor b

4. Prediction vs. Explanatory • Predictive research is aimed at predicting outcomes from a collection of variable • Explanatory research is aimed at testing hypotheses formulated to explain phenomena of interest • Major difference is the role of theory. Explanatory research usually lacks a theoretical framework and it is a search to uncover independent variables

5. Logic of Comparison • Experimental Designs • comparison made among groups that, because of randomization, are equal across all things except for the “treatment” • Quasi-experimental Designs • comparison made among groups that have been exposed to different “treatments” but are groups that are often not like to begin with

6. Logic of Comparison • Nonexperimental designs • Often grouped based on the dependent variable… e.g. persisters/nonpersisters, users of technology, nonusers of technology… • We then try to uncover the variables that have “caused” the observed differences in groups. • So we have group problems: • they are not alike across the independent variables • they may have been exposed to the same treatment • Sampling and design are key. In nonexperimental research we need to be very careful of the results and implications we put forth based on our samples and designs

7. Control vs. Comparison Groups • The major threat to validity in nonexperimental research comes from uncontrolled unmeasured covariates… confounding variables

8. Categorical Independent Variables • Used when we classify people into groups and are interested in group differences • Unless a categorical variable is a “treatement”, there is no causation implied… in other words a difference between males and females on an outcome becomes a question, not an answer. • What is it about males and females that make them differ on the phenomena being studied?

9. How We Treat Categorical IV’s • For Continuous Outcomes • Dichotomous independent variables • t-tests • Polychotomous independent variables • ANOVA or multiple regression • For nominal outcomes • crosstabs, Chi-Square

10. Coding Categorical IV’s • We are familiar with the dichotomous independent variable sex: female/male • We have also been using race: students of color/white • Reminder: the groups must be mutually exclusive • We still stick to Dummy Coding in this class. (Behind the scenes, SPSS is doing effects coding)

11. Dummy Coding • Consists of 1s and 0s with 1 signifying membership in a category and 0 signifying nonmembership • Let’s extend our dichotomous IV to n levels. You will have n columns representing the n groups but will only use n-1 groups in a regression model.

12. Example With Data RELIGION J C M O 1 Jewish 1 0 0 0 2 Christian 0 1 0 0 3 Muslim 0 0 1 0 4 Other 0 0 0 1 5 Muslim 0 0 1 0 6 Jewish 1 0 0 0 Four columns represent four mutually exclusive groups. In this example, the first and sixth subjects are Jewish

13. Using Categorical IV’s • In our example, we only need three of the columns to correctly specify a contrast between two groups • The group you leave out of the regression, will become your “reference category” • A reference category is technically represented by the constant • Contrasts will be between the three included groups and the group you left out

14. Our Example • Regression • Dependent Variable: Promote Racial Understanding • Independent Variable: Religion • If I include C, M and O as variables and leave out J. Then I will have three variables representing three contrasts • C-the difference between Christians and Jewish • M-the difference between Muslims and Jewish • O-the difference between Others and Jewish • If we want a different set of contrasts we choose a different group as the referent group to leave out

15. Statistical Control • Forms of control • manipulation, elimination/inclusion, statistical, randomization • Manipulation—the control the researcher has over manipulation of a treatment • Elimination/Inclusion—either we eliminate by holding them constant (only study females) or we include them so they can be estimated • Statistical—include them as covariates, control for them but not interested in them • Randomization—with random assignment, we control for observed and unobserved covariates!!

16. Statistical Control inOur Context • Rather than holding IVs constant through experimental control, influence is held constant by statistical techniques (by removing influence of confounding factors) • Application to non-experimental designs makes causal interpretations difficult • Measurement concerns are important for the IVs

17. ANCOVA • The age old question: “What is the difference between ANCOVA and Regression?” • Those trained in experimental research are usually taught to apply and “speak” in ANCOVA terms • Those trained in quasi-experimental, correlational research are usually taught to apply and “speak” in Regression terms

18. ANCOVA • They are parallel analytical techniques. One usually employs ANCOVA in cases where the “treatment” is manipulated and of a causal nature. • The field of higher education primarily utilizes Multiple Regression. • Doing ANCOVA through a multiple regression program not only enables one to see clearly what is taking place but also affords the control necessary to carry out the analysis required. (Pedhazur & Pedhazur, 1991, p. 568).

19. Theoretical: Interactions • Without interactions between predictors in the model, we assume a constant effect for all levels of each independent predictor • Interactions allow the effects of variables depend on the level of OTHER independent variables. • Example: the effect of race for promoting racial understanding differs for genders implies an interaction between sex and race

20. Interactions • Ordinal—The regression lines do not intersect within the range of another independent variable (the rank order of the effect does not change) • Disordinal—The regression lines intersect within the range of another independent variable (the rank order of the effect changes, flips)

21. Graphical Interactions or Lack Of

22. ANCOVA Example • Reminder: Including variables as statistical controls reduces the error variance thus increases the sensitivity of the analysis. • Our example: • dependent—promote racial understanding • independents—attend cultural awareness workshop • controls—sex, race • interaction—race*cultural awareness workshop

23. Assumptions • All the normal regression assumptions about normality, homogeneity, independence • Assume effect of attending a cultural awareness workshop is constant across males and females • Allow the effect of attending a cultural awareness workshop to vary across whites/students of color by adding the interaction

24. Results Note: 21% of students attended a workshop, attending at approximately equal rates across the race groups

25. Results

26. Intepretation • The interaction is not significant so we can say: • the effect of attending a cultural awareness workshop is constant across whites vs. students of color • all three effects, sex, race, workshop are significant • females, students of color and those attending workshops are more likely to believe promoting racial understanding is important

27. Graphing Interactions

28. Last But Not Least • Adusted Means • We do this by plugging values • White, Male, Attend=1.99 • White, Male, No Attend=1.41 • SOC, Male, Attend=2.86 • SOC, Male, No Attend=2.12

29. For Next Week • Read Pedhazur Ch 3 32-39 • Read Pedhazur Ch 4 p66-70 • Read Pedhazur Ch 22 p590-606