Quasi-Experimental Designs

1. 10 Quasi-Experimental Designs

2. Introduction • Quasi-experimental design • a research design in which an experimental procedure is applied, but all extraneous variables are not controlled • typically lacking random assignment

3. Introduction • Causal inferences are made by ruling out rival hypothesis • identification and study of plausible threats to internal validity • control by design • additional pretests time points • additional control groups • coherent pattern matching • making a complex prediction that few rival hypotheses can explain

4. Nonequivalent Comparison Group Design • The most common quasi-experimental design • Includes both an experimental and control group • Participants not randomly assigned to groups • Pre-test important to determine equivalence of groups • large difference between groups on pre-test may indicate selection bias • Threats frequently reveal themselves in the outcome

6. Outcomes with Rival Hypotheses • Increasing treatment and control groups • greater increase in treatment condition • could be caused by a number of rival hypotheses • selection-maturation • selection-history • other selection-interactive effects

7. Outcomes with Rival Hypotheses • Experimental group higher than control group at pre-test effect • no change in control group • treatment condition starts higher and increases • rival hypotheses • selection-history

8. Outcomes with Rival Hypotheses • Experimental group lower than control group at pre-test effect • no change in control group • treatment condition starts lower and increases • rival hypotheses • selection-regression

9. Outcomes with Rival Hypotheses • Crossover effect • treatment group scores significantly lower than the control group at pretest • significantly higher at posttest • no change in control group • rival hypotheses unlikely with this type of result

10. Ruling out Threats to the Nonequivalent Comparison Group • Matching • equates the groups on important variables • example • Head Start programs can be matched on income, intelligence, and parental involvement • matching not a perfect replacement for random assignment, but can be used when random assignment is not possible • selection-regression effects may occur when using extreme groups • Statistical control techniques • ANCOVA

11. Causal Inference from Nonequivalent Comparison Group Design • To increase internal validity • do not let participants self-select into groups • self-selection increases bias • minimize pretest differences in groups • matching • ANCOVA during data analysis

12. Interrupted Time-Series Design • Design in which a treatment effect is assessed by comparing the pattern of pre- and posttest scores for a single group of research participants • Looking for discontinuity in the series of dependent measures • Example • Lewis and Eves (2012)

13. Interrupted Time-Series Design • Use of multiple pretest and posttest measurements demonstrates reliability of effect • Visual inspection of pre and posttest measure very important to determine treatment effect • Improvement over one-group pretest-posttest design • Primary weakness • no control of history effects

14. Regression Discontinuity Design • Design that assigns participants to groups based on their scores on an assignment variable and assesses the effect of a treatment by looking for a discontinuity in the groups regression lines

15. Regression Discontinuity Design • Characteristics of the design • all individuals are pretested • individuals who score above some cutoff score receive the treatment • all individuals are posttested • discontinuity in the regression line indicates a treatment effect

16. Regression Discontinuity Design