Download
1 / 28
280 likes | 290 Views
Learn about experimental design in research, including randomization, control groups, manipulation of exposure, and threats to internal validity. Understand how to enhance internal validity through random assignment and minimize selection bias, history, maturation, testing, and attrition.
E N D
HDFS 361—Research Methods Experimental and Quasi-Experimental Design
What is an Experiment? Experiments, be they qualitative or quantitative, involve three essential features: • Two or more groups or treatment conditions • Random assignment of participants to each of these groups • Exposure of each group to a different level of an independent variable or to different levels of an intervention
Randomization is a process whereby members of a sample (i.e., individual participants) are randomly distributed throughout experimental conditions or groups. Randomization assures that qualities of individual participants that might somehow influence the outcome of an experiment are not systematically concentrated in any particular experimental condition. Randomization
Control groups are experimental conditions where participants are not exposed to independent variables of interest or are exposed at very different levels than are subjects in treatment groups. Treatment groups are experimental conditions where participants are exposed to independent variables of interest. The exposure to the independent variable is manipulated. Groups and Manipulation of Exposure
Randomization of Maria’s Participants to Experimental Conditions • Maria begins by listing her 40 students in alphabetical order and assigning each of them a number from 1 to 40. If she had 50 students, she would use numbers 1 to 50. She lists students’ names in alphabetical order, but any order is acceptable because random assignment to groups is made on the basis of random numbers. • She then lists the numbers she assigned to individuals in order (without the names) as shown in the left two columns of Figure 1.1.
Randomization of Maria’s Participants to Experimental Conditions • Next, she begins the process of random assignment to groups by randomly selecting 20 students to be in the treatment group. • She selects numbers at random beginning at any point in Appendix A’s table of random numbers. The first number between 1 and 40 is the number 12 so she records a 12 in the box labeled Treatment Group. This means the 12th person is going to be assigned to the treatment group and we put a check by participant number 12. Continuing down the column, the next number in the range of 1 to 40 is the number 29, so she checks this number and writes it under the control group.
+-----+ | id | |-----| 1. | 118 | 2. | 156 | 3. | 182 | 4. | 180 | 5. | 57 | |-----| 6. | 12 | 7. | 73 | 8. | 174 | 9. | 29 | 10. | 8 | |-----| 11. | 86 | 12. | 161 | 13. | 71 | 14. | 51 | 15. | 129 | First 15 numbers from Appendix A
Internal Validity • A experiment’s internal validity refers to its ability to rule out the influence of extraneous variables on the dependent variable of interest. • A study with high internal validity eliminates threats from extraneous variables such that any influence on the dependent variable can be assumed to proceed from the independent variable. • For example, randomization of participants to groups strengthens internal validity by insuring that groups are not systematically different on extraneous variables.
Internal Validity—Selection Bias • Selection bias refers to how participants are selected into treatment and control groups in an experiment. • Whenever any source of bias operates in the process of composing treatment and control groups in a study, that bias threatens the internal validity of the study.
Internal Validity—History • The threat of history is the possibility that an external event occurring during the course of an experiment may influence how participants score on dependent measures. • For example, a natural disaster could happen during the course of the study.
Internal Validity—Maturation • The threat of maturity regards the possibility that an internal change participants may go through during the course of an experiment may somehow influence how they score on dependent measures. • Adolescence is a period where this can be a serious problem.
Internal Validity—Testing • The threat of testing regards the possibility that a participant’s experience taking pretest measures influence how they score on posttest measures. • A pretest measuring knowledge of the adverse effects of smoking may sensitize people to these risks and be the real reason they quit.
Internal Validity—Mortality/Attrition • The threat of mortality or attrition regards the possibility participants may drop out of a study for some systematic reason, leaving only participants with certain qualities to take dependent measures. • Program to reduce delinquency may have the more delinquent participants quit and those who are left make the program look better than it is.
Internal Validity—Statistical Regression • The threat of statistical regression toward the mean is the possibility that a subject who scores extremely high or low on a pretest measurement of a dependent variable may naturally score closer to the mean for their group at the posttest measurement of the same variable. • The difference between pretest and posttest scores for such a subject may not reflect the influence of an independent variable on the dependent variable so much as it may reflect the natural tendency for people who achieve extreme scores at one point in time to achieve less than extreme scores at a later time.
External Validity • An experiment’s external validity refers to the ability to generalize beyond the study to a larger population, realistic circumstances, and other periods of time. • A study with high external validity will guarantee that what it found to be true for its participants should be true for the population of people those participants were meant to represent. • For example, a random sample of participants to include in the study strengthens external validity by insuring that the participants are not systematically different from the population.
External Validity—Sampling Bias • Sampling bias refers to the possibility that the sample of participantsin an experiment is somehow biased. • An unbiased sample would accurately mirror the population it is meant to represent. • Ideally the pool of participants in a study are a probability sample of a larger population.
External Validity—Environmental Validity • An experiment has a high level of environmental validity when the conditions and circumstances of the experiment accurately mirror the reality to which the study is generalized • Many studies are done in artificial situations and these do not generalize to the real world
External Validity—Demand Characteristics • Demand characteristics of a study threaten external validity by influencing responses of participants. These characteristics essentially compromise the ability of a sample to accurately represent the population from which it was drawn • A double blind procedure, where staff who interact with study participants are not told the research hypothesis or which group participants are in, can mitigate demand characteristics
Experimental DesignsPosttest-Only Treatment and Control Group
Experimental DesignsPretest-Posttest Treatment and Control Group
