Introduction to Research Design

1. Introduction to Research Design Threats to Internal Validity One Group Pretest-Posttest Design

2.  O   X   O • Campbell & Stanley “pre-experimental.” • Wuensch “experimental.” • OK design if can achieve “experimental isolation,” as in the chemistry lab. • Correlated samples t test or nonparametric equivalent.

3. History Events other than X between pretest and posttest • Pre = subjects’ energy consumption. • X = Education on importance of conserving resources. • Post = subjects’ energy consumption. • History = price of energy increases 50% between pre and post.

4. Maturation Processes that normally cause subjects to change across time. • Subjects = newly hired employees • Pre = Test of morale • X = Six month program to elevate morale • Post = Test of morale • Maturation = end of honeymoon effect

5. Testing Pretesting subjects can change them. • Pre = frequency of conservation behaviors • have you installed a low-flow shower head? • et cetera • X = Education on importance of conserving resources • Post = frequency of conservation behaviors • Testing = just (pre) asking them about certain behaviors might cause them to try them.

6. Instrumentation The measuring instrument changes across time. • The $1.99 scale for our fishing experiment. • AM versus PM weight of our catch • Spring stretched • The human observer as instrument • effect of treatment on number of problems in computer lab • changes in observers from pre to post

7. Statistical Regression When scores have an error component, both high and low scores regress towards the mean upon retesting. • My ESP demo in PSYC 2101 • Educational research at Miami Univ. • mean IQ in school district = 102 • mean of selected students = 80 • how much regression is expected?

8. Percent of regression towards the mean, PRM = 100(1 – ρ). • If pretest-posttest corr = .8, • PRM = 100(1 - .8) = 20%. • Expect regression (up) of .2(102 - 80) = 4.4 points. • That may be enough to get significant pre-post change.

9. Mortality Subjects who drop out of the experiment may differ from those who stay in. • Subjects = patients with wasting disease • Pre, Post = body weight, X = New drug • 20 patients at pre, 10 at post • Pre mean weight = 97 lb, post = 125. • Was the drug effective?

10. Perhaps the sickest (and lightest) 10 dropped out. • Maybe positive effect from some, who stay in, negative for others, who drop out. • Correct by computing means only for those who complete study. • creating a problem of external validity.