Non-Experimental designs: Correlational & Quasi-experimental designs

Non-Experimental designs: Correlational & Quasi-experimental designs Psych 231: Research Methods in Psychology

Sometimes you just can’t perform a fully controlled experiment • Because of the issue of interest • Limited resources (not enough subjects, observations are too costly, etc). • Surveys • Correlational • Quasi-Experiments • Developmental designs • Small-N designs • This does NOT imply that they are bad designs • Just remember the advantages and disadvantages of each Non-Experimental designs

Looking for a co-occurrence relationship between two (or more) variables • Used for • Descriptive research • do behaviors co-occur? • Predictive research • is one behavior predictive of another? • Reliability and Validity • Does your measure correlate with others (and itself)? • Evaluating theories • Look for co-occurrence posited by the theory. Correlational designs

Looking for a co-occurrence relationship between two (or more) variables • Example 1: Suppose that you notice that the more you study for an exam, the better your score typically is. • This suggests that there is a relationship between study time and test performance. • We call this relationship a correlation. • 3 properties: form, direction, strength Correlational designs

Y 6 5 4 3 2 1 1 2 3 4 5 6 X Scatterplot

Linear Non-linear Form

Y Y X X Positive Negative • X & Y vary in the same direction • X & Y vary in opposite directions Direction

r = 1.0 “perfect positive corr.” r = 0.0 “no relationship” -1.0 0.0 +1.0 The farther from zero, the stronger the relationship r = -1.0 “perfect negative corr.” Strength

r = -0.8 r = 0.5 -.8 .5 -1.0 0.0 +1.0 Rel A Rel B • Which relationship is stronger? • Rel A, -0.8 is stronger than +0.5 Strength

Y 6 5 4 3 2 1 1 2 3 4 5 6 X • For our example, we have a linear relationship, it is positive, and fairly strong Scatterplot

Looking for a co-occurrence relationship between two (or more) variables • Explanatory variables (Predictor variables) • Response variables (Outcome variables) • Example 1: Suppose that you notice that the more you study for an exam, the better your score typically is • For our example, which variable is explanatory and which is response? And why? • It depends on your theory of the causal relationship between the variables Correlational designs

Y 6 5 4 3 2 1 1 2 3 4 5 6 X Response (outcome) variable • For descriptive case, it doesn’t matter which variable goes where • Correlational analysis • For predictive cases, put the response variable on the Y axis • Regression analysis Explanatory (predictor) variable Scatterplot

Advantages: • Doesn’t require manipulation of variable • Sometimes the variables of interest can’t be manipulated • Allows for simple observations of variables in naturalistic settings (increasing external validity) • Can look at a lot of variables at once Correlational designs

Disadvantages: • Don’t make casual claims • Third variable problem • Temporal precedence • Coincidence (random co-occurence) • Correlational results are often misinterpreted Correlational designs

Disadvantages: • Example 2: Suppose that you notice that kids who sit in the front of class typically get higher grades. • This suggests that there is a relationship between where you sit in class and grades. Daily Gazzett Children who sit in the back of the classroom receive lower grades than those who sit in the front. Possibly implied: “[All] Children who sit in the back of the classroom [always] receive worse grades than [each and every child] who sits in the front.” Better: “Researchers X and Y found that children who sat in the back of the classroom were more likely to receive lower grades than those who sat in the front.” Misunderstood Correlational designs Example from Owen Emlen (2006)

What are they? • Almost “true” experiments, but with an inherent confounding variable • General types • An event occurs that the experimenter doesn’t manipulate • Something not under the experimenter’s control • (e.g., flashbulb memories for traumatic events) • Interested in subject variables • high vs. low IQ, males vs. females • Time is used as a variable Quasi-experiments

Advantages • Allows applied research when experiments not possible • Threats to internal validity can be assessed (sometimes) • Disadvantages • Threats to internal validity may exist • Designs are more complex than traditional experiments • Statistical analysis can be difficult • Most statistical analyses assume randomness Quasi-experiments

Program evaluation • Research on programs that is implemented to achieve some positive effect on a group of individuals. • e.g., does abstinence from sex program work in schools • Steps in program evaluation • Needs assessment - is there a problem? • Program theory assessment - does program address the needs? • Process evaluation - does it reach the target population? Is it being run correctly? • Outcome evaluation - are the intended outcomes being realized? • Efficiency assessment- was it “worth” it? The the benefits worth the costs? Quasi-experiments

Independent Variable Dependent Variable Dependent Variable Non-Random Assignment Experimental group Measure Measure participants Control group Measure Measure • Nonequivalent control group designs • with pretest and posttest (most common) (think back to the second control lecture) • But remember that the results may be compromised because of the nonequivalent control group (review threats to internal validity) Quasi-experiments

Interrupted & Non-interrupted time series designs • Observe a single group multiple times prior to and after a treatment Obs Obs Obs Obs Treatment Obs Obs Obs Obs • Look for an instantaneous, permanent change • Interrupted – when treatment was not introduced by researcher, for example some historical event • Variations of basic time series design • Addition of a nonequivalent no-treatment control group time series O O O T O O O & O O O _ O O O • Interrupted time series with removed treatment • If treatment effect is reversible Quasi-experiments

Non-Experimental designs: Correlational & Quasi-experimental designs