Education 795 Class Notes

1. Education 795 Class Notes Quasi-Experimental Design Path Analysis Note set 9

2. Today’s Agenda • Announcements (ours and yours • Q/A • Quasi-experimental design • Path analysis as an approach to variance partitioning and causal modeling

3. Quasi-Experimental Design • Historical Happenings • Statistical analyses evolved to meet needs of experimental designs • Quasi-experimental designs evolved in the social sciences • Researchers continue to used experimental analyses for quasi and non experimental designs • Statistical analyses emerged to meet the challenges of quasi and non experimental designs • Researchers adopt new and improved techniques • New statistical analyses continue to emerge to meet the many challenges that quasi and non experimental designs face… • non-random assignment • no manipulation of treatment

4. Nomenclature • Quasi-experimental designs refer to studies where no random assignment is in place • We cannot separate the irrelevant causal forces hidden within the ceteris paribus of random assignment (Cook & Campbell, 1979). • Refer to quasi-experimental when there is a treatment in place but no random assignment and we are interested in ‘causal effects’. • Refer to non-experimental when we want to explain differences among groups

5. The Term “Treatment” • Treatments can be: • interventions • direct manipulation of a variable • naturally occurring • abrupt and precisely dated • training programs • exposure to a condition

6. X X Y Y e e The Omitted Variables Conundrum A B A Y When the error is correlated with the treatment (X) we cannot separate out the “treatment” effect from spurious effects

7. How Do We Deal With This Problem to Get at Causation? • Return to a regression-based approach, and introduce a special kind of regression called Path Analysis • Introduce Structural Equation Modeling

8. Review Regression Formula • Raw score depiction: where each b: • is the unique and independent contribution of that predictor to the model • for quantitative IVs, the expected direction and amount of change in the DV for each unit change in the IV, holding all other IVs constant • For dichotomous IVs, the direction and amount of group mean difference on DV, holding all other IVs constant

9. Review Venn Diagram Regression Coefficients Correlation

10. Venn Diagrams Partial Correlation Correlation

11. Relationships among 3 variables A. C. B B A A S S D. B B. B A A S S

12. Path Analysis • A structured approach to regression analysis allowing intervening variables • Nomenclature • Path Analysis • LISREL Models • Causal Models • ANCOVA Models • Latent Variable Models • Structural Equation Models

13. Causation • Most controversial topic in philosophy of science and has been characterized as ‘a notorious philosophical tar pit’ (Davis, 1985, p. 8) • The history of the topic extends over centuries

14. Random Selection of Quotes • ‘Cause is the most valuable concept in the methodology of the applied sciences’ (Scriven, 1968, p. 79) • ‘Let’s drop the word cause and bring educational research out of the middle ages’ (Travers, 1981, p. 32) • ‘It would be very healthy if more researchers abandon thinking of and using terms such as cause and effect’ (Muthien, 1987, p. 180) • No causation without manipulation (Holland & Rubin, 1986)

15. Return to the Role of Theory • Causal analyses are based on theory. • The temptation to apply sophisticated state-of-the-art methodologies seem irresistable • It is important to recognize when a given methodology is inapplicable • ‘In sum, the formulation of a causal model is an arduous and long process entailing a great deal of critical thinking, creativity, insight and erudition’ (Pedhazur & Pedhzur, 1991, p. 699)

16. Definitions • Exogenous Variable– variable with arrows ONLY going out of it in the model (Strictly predictor) • Endogenous Variable—variable with arrows going IN—it can also have arrows going out (Outcome and possibly a mediated predictor)

17. Definitions • Direct Effect—the effect of a variable that has a direct path to the outcome. • Indirect Effect—the effect of a variable on an outcome that travels through (is mediated by) other variables in the model • Total Effect—Sum of the direct and indirect effects for one variable on the outcome

18. Definitions • X, Z predictors, Y, outcome • Spuriousness • The relationship between X and Y is said to be spurious if Z causes X and Y • Unexplained covariation • Both X and Y are exogenous and so variation between them is not explained by the model

19. Example: Understanding the Effects of Frog Ponds • Theoretical discussion started by odd findings related to student achievement • Reference-group theory: Environmental press or relative deprivation? • Is it better to be a small frog in a big pond, or a big frog in a small pond?

20. Path Analysis Example • Bassis model • With respect to academic self-evaluation, is it better to be a small frog in a big pond, or a big frog in a small pond?

21. Bassis Model

22. Descriptive statistics

23. Bassis Results

24. Decomposition of r • Correlation between two endogenous variables: r = Direct Effect + Indirect Effects + Spuriousness • Correlation between an endogenous variable and an exogenous variable: r = Direct Effect + Indirect Effects + Unspecified Covariance

25. Calculating Path Coefficients • Compute the appropriate regression analyses, and organize the resulting coefficients • First, calculate the indirect components by multiplying the involved coefficients • Second, sum the indirect components and add to the direct coefficient (if any) to calculate the total effect

26. Computing Exercise Calculate the direct and indirect effects of: a) Selectivity on 4-year academic self-rating b) HS grades on college grades

27. Comparing Models • One interesting use of path analysis is to directly compare models • Higher education as a contextualized experience:Are frog pond effects similar in majority / specialized institutions for students served by such specialized institutions?

28. African American Students inPWIs and HBCUs PWIs HBCUs

29. Women Students in Coed and Single-sex Institutions Coed Womens’colleges

30. For Next Week • Read Pedhazur Ch 7 p 152-157 • Read Aldrich & Nelson, ALL