Questions

1. Questions • What is the best way to avoid order effects while doing within subjects design? • We talked about people becoming more depressed during a treatment period, but would a history effect also include if a participant became happier due to an outside factor other than the treatment being done? • What is an example of participant attrition? • When a study is being conducted, if a confounding variable was found, would the study make changes to control the confounding variable right away, or would they continue the study but note the confounding variable in the discussion?

2. More Questions • How do you determine how much time should pass between treatments to ensure that your counterbalancing is effective? • I really don’t understand counterbalancing. Can you explain it in a really simplistic way?? Why would you use this method? • Two groups going through the same treatments in different order – it is still a within subjects design • Does counterbalancing eliminate order effects? • In matched-subject designs is the matching randomly assigned even though researchers are matching participants based on certain characteristics?

3. More Questions • Is it possible to use other statistical analyses such as regression, Chi-square, F-test etc. rather than ANOVA and t-test in within-subjects or between-subjects designs? • Regression – determines function of the best fit to your data (relationship between dependent and independent variables) • Chi-square – data consisting of proportions • F-test (ANOVA) – t-test (two groups) as a special form of F-test (more than two groups) • Analysis of covariance – ANCOVA • Multivariate analysis of variance - MANOVA • Can the final be open notes? I wrote out all the key words, did all the exercises etc. I can’t memorize like that. Are you going to curve?

4. Nonexperimental and Quasi-Experimental Strategies: Nonequivalent Group, Pre-Post, and Developmental Designs Chapter 10 Dusana Rybarova Psyc 290B May 26 2006

5. 1. Introduction • quasi-experimental research strategy • like an experiment, typically involves a comparison of groups or conditions • however, it uses a nonmanipulated variable to define groups or conditions being compared such as age, gender or time (before vs. after treatment) • within the context of quasi-experimental research, the variable that is used to differentiate the groups of participants or the groups of scores being compared is called the quasi-independent variable (e.g. age) • the variable that is measured to obtain a score for each individual is called the dependent variable (e.g. IQ score)

6. 1. Introduction • Nonexperimental research strategy • Very similar to quasi-experimental research strategy • Major distinction – nonexperimental designs make little or no attempt to minimize threats to internal validity • Just like in the quasi-experimental designs there is no real manipulation of variables • In contrast to quasi-experimental designs there is less rigor in control of extraneous variables

7. 1. Introduction • in what follows we will use the following symbols • X – represents the treatment • O – represents an observation or measurement • 2 groups of nonexperimental and quasi-experimental designs: • Between-subjects designs or nonequivalent group designs • Within-subjects designs or pre-post designs

8. 2. Nonequivalent group designs (between-subjects) • nonequivalent group design is a research study in which the different groups of participants are formed under circumstances that do not permit the researcher to control the assignment of individuals to groups because those groups are pre-existing; the groups of participants are therefore considered nonequivalent • e.g. a researcher wants to evaluate a teen pregnancy prevention program by comparing the pregnancy rates in a high school where the program is used with pregnancy rates in a high school that does not use the program

9. 2. Nonequivalent group designs • nonequivalent group design has a built-in threat to internal validity that precludes an unambiguous cause-and-effect explanation, i.e. assignment bias • three examples of nonequivalent group designs • the differential research design (nonexperimental) • the posttest-only nonequivalent control group design (nonexperimental) • the pretest-posttest nonequivalent control group design (quasi-experimental)

10. 2. Nonequivalent group designs • the differential research design (nonexperimental) • simply compares pre-existing groups • uses a participant characteristic such as gender, race, or personality to automatically assign participants to groups • no random assignment of participants to groups • dependent variable is then measured for each participant to obtain a set of scores within each group • the goal of the study is to determine whether the scores for one group are consistently different from scores in another group • e.g. mother’s and father’s involvement in the peer relationships of their adolescent children • in differential research, participant differences in one variable are used to create separate groups, and measurements of the second variable are made within each group • correlational study treats all the participants as a single group and simply measures the two variables for each individual

11. 2. Nonequivalent group designs • Post-test only nonequivalent control group design (nonexperimental) • compares two nonequivalent groups of participants • one group is observed (measured) after receiving a treatment, and the other group is measured at the same time but receives no treatment • Does not protect agains assignment bias • e.g. the teen pregnancy program X O (treatment group) O (nonequivalent control group)

12. 2. Nonequivalent group designs • Pretest-posttest nonequivalent control group design (quasi-experimental) • compares two nonequivalent groups • one group is measured twice – once before a treatment and once after • Problem of assignment bias even though it is reduced by pre and post measurement • Potential problems with differential history effects, differential instrumentation, differential testing effects, differential maturation and differential regression • the other group is measured at the same two times but does not receive any treatment O X O (treatment group) O O (nonequivalent control group)

13. 3. Pre-post designs (within-subjects) • series of observations is made over time on one group of subjects • Internal validity is threatened by time-related effects • one-group pretest-posttest design (nonexperimental) • Each individual in a group is measured once before and once after a treatment • Does not control for possible extraneous variables possibly causing change over time O X O e.g. evaluating the effectiveness of a new political TV commercial

14. 3. Pre-post designs (within-subjects) • time series and interrupted time-series designs (quasi-experimental) • Series of observations for each participant before a treatment and a series of observations after the treatment • Eliminates many problems with the pretest-postest design, outside events are a confound only if they occur simultaneously with the treatment O O O X O O O • Time series design – treatment administered by the researcher • e.g. anger management program for students • Interrupted time-series design – event or treatment is not created by the researcher • e.g. legal change

15. 3. Pre-post designs (within-subjects) • Equivalent time-sample design (quasi-experimental) • Consists of a long series of observations during which a treatment is alternately administered and then withdrawn OOOXOOONOOOXOOONOOOXOOON… N – no treatment • It reduces likelihood of an external event confound to the treatment • Differences between expected temporary and permanent effects

16. 4. Developmental research designs • the purpose is to describe the relationship between age and other variables • Cross-sectional research design (nonexperimental) • uses different groups of individuals, each group representing a different age • the different groups are measured at one point in time and then compared (e.g. IQ in 30, 40, 50 and 60 years old) • individuals who were born at roughly the same time and grew up under similar circumstances are called cohorts • the terms cohort effect or generation effect refer to a difference between age groups (or cohorts) due to unique characteristics or experiences other than age; generation effects are a type of third-variable problem

17. 4. Developmental research designs • Longitudinal research design (nonexperimental) • examines development by making a series of observations or measurements over time • commonly the same group of individuals is followed and measured at different points in time • Avoids problems with cohort effects but is usually very time-consuming and expensive • Serious problem with participant attrition • e.g. follow-up on IQ changes in the same group of subjects when they are 20 years old, 30, 40 and 50 years old