330 likes | 410 Views
GROUP-LEVEL DESIGNS. Chapter 9. CHARACTERISTICS OF “IDEAL” EXPERIMENTS. Research designs that can establish a causal relationship between variables Six characteristics Time order of the independent variable (IV) IV is manipulated The IV and DV have a relationship
E N D
GROUP-LEVEL DESIGNS Chapter 9
CHARACTERISTICS OF “IDEAL” EXPERIMENTS • Research designs that can establish a causal relationship between variables • Six characteristics • Time order of the independent variable (IV) • IV is manipulated • The IV and DV have a relationship • Rival hypotheses are controlled for • At least one control group is used • Random assignment is used
Controlling the Time Order of Variables • The Independent Variable (IV) must occur before change in the dependent variable (DV) is observed
Manipulating the Independent Variable • Three ways to manipulate the IV • One group experiences the IV (X is present) and one group does not (X is absent) • Adjust the “dosage” or amount of exposure of the IV (small amount of X versus large amount of X) • One group experiences the IV (X is present) and one group experiences an alternative intervention (something else)
Establishing Relationships between Variables • The relationship between the independent and the dependent variables must be established in order to infer a cause-effect relationship
Controlling Rival Hypotheses • Rival hypotheses – a plausible alternative explanation to the research hypothesis • Three ways to rule out other extraneous variables that might affect the dependent variable • Holding Extraneous Variables Constant • Using Correlated Variation • Using Analysis of Covariance
Holding Extraneous Variables Constant • Any extraneous variables that might affect the dependent variable are held constant in an ideal experiment by • Being applied equally to all research participants • If gender is thought to affect the DV, then limit the sample to only females • Remaining unchanged for the duration • If dosage of medication is thought to affect the DV, then keep the dosage constant
Using Correlated Variation • If several independent variables are used in a research study, then determine to what degree they are correlated • If the correlation between two independent variables is high, then only one of those variables needs to be included
Using Analysis of Covariance • A statistical method that is used to compensate for differences between the groups being compared in a research study
Using a Control Group • The group in the research study that does not receive the intervention (or IV) • A control group is only effective when research have been randomly assigned to either the experimental or the control group • Symbols • R = Randomization (selection or assignment) • O = Observation (measurement) of the DV • X = Independent variable or IV
Randomly Assigning Research Participants to Groups • After research participants have been randomly selected from the population, they are randomly assigned to either an experimental or control group • Research participants are assigned to either group on the basis of chance (they have an equal chance of being in the experimental or control group)
Matched Pairs • Another method of dividing research participants into comparison groups • Research participants are matched on key characteristics, then one individual from each pair is place into two separate groups • Individuals that do not have a pair are eliminated from the study
INTERNAL AND EXTERNAL VALIDITY • Group-level designs are evaluated based on their ability to generate knowledge • Internal Validity – the degree to which a research design can ensure that the independent variable is the sole cause of change in the dependent variable • External Validity – the extent to which the findings of a research design can generalized to other groups (population) or situations
History Maturation Testing Instrumentation Error Statistical Regression Differential Selection Mortality Reactive Effects Interaction Effects Inter-group Relations Threats to Internal Validity (Box 8.1) • Known threats that provide alternative explanations (rival hypotheses) for what might bring about change in the dependent variable
Threats to External Validity (Box 8.2) • Known threats that limit or restrict the degree to which research study results are generalizable • Pretest-Treatment Interaction • Selection-Treatment Interaction • Specificity of Variables • Reactive Effects • Multiple Treatment Interference • Researcher Bias
GROUP RESEARCH DESIGNS • Group research designs are categorized along the continuum of knowledge • Exploratory Designs • Descriptive Designs • Explanatory Designs
Exploratory Designs • Do not contain any of the requirements of the “ideal” experiment • Threats to internal and external validity are high (i.e., virtually all apply) • These designs are used to explore a research question about which little is already known in order to uncover generalizations and to develop hypotheses for further investigation and testing
One-Group Posttest-Only Design • Involves a single measure or observation (O1) of the dependent variable that occurs after one group of people has experienced the intervention (X) • Design Blueprint: X O1 • The design does not control for any threats to internal or external validity
Cross-Sectional Survey Design • Another form of a one-group posttest-only design but the intervention (X) is not specified • A cross-section of a population is observed (O1) at one particular point in time • Design Blueprint: O1
Multigroup Posttest-Only Design • The one-group posttest-only design is applied to multiple groups • A single measure or observation (O1) of the dependent variable occurs after each group experiences some form of the intervention (X) • Design Blueprint: Group 1: X O1 Group 2: X O1
Longitudinal Case Study Design • Involves repeated measures or observations (O1) of the dependent variable that occurs after one group of people has experienced the intervention (X) • Design Blueprint: X O1 O2 O3
Longitudinal Survey Design • Involves repeated measures or observations (O1) of one group of people over time • Design Blueprint: O1 O2 O3 • Trend Studies – repeated observations on multiple samples drawn from one population • Cohort Studies – repeated observations on one group (sample)
Descriptive Designs • Apply some “Ideal” experiment features: • Time order of variables • Manipulation of the independent variable • Use of comparison group (not a control group) • Random selection but not random assignment • Compared to exploratory designs, threats to internal and external validity are reduced
Randomized One-Group Posttest-Only Design • Involves a single measure or observation (O1) of the dependent variable that occurs after one randomly selected group of people has experienced the intervention (X) • Design Blueprint: R X O1
Randomized Cross-Sectional Survey Design • Another form of a randomized one-group posttest-only design but the intervention (X) is not specified • A randomly selected cross-section of a population is observed (O1) at one particular point in time • Design Blueprint: R O1
One Group Pretest-Posttest Design • Involves repeated measures or observations of the dependent variable • The first observation (O1) occurs before the intervention (X) and the second observation (O2) occurs after it • The posttest (O2) is compared to the pretest (O1) to determine if any change occurred in the dependent variable • Design Blueprint: O1X O2
Comparison Group Posttest-Only Design • Involves two groups • The “experimental” group receives the intervention (X) while the comparison group does not • A single measure (O1) of the dependent variable occurs for each group • Design Blueprint: Group 1: X O1 Group 2: O1
Comparison Group Pretest-Posttest Design • Involves repeated measures or observations of the dependent variable on two groups • An “experimental” and a comparison group • The first observation (O1) occurs before the experimental group receives X and the second observation (O2) occurs after X • Design Blueprint: O1X O2 O1 O2
Interrupted Time-Series Design • Involves repeated measures or observations of the dependent variable on one group • Several observations occur before the intervention (X) and several occur after • Design Blueprint: O1O2O3X O4 O5 O6
Explanatory Designs • Most closely approximate (and include) the “ideal” experiment • Most threats to internal and external validity are eliminated or “ruled out” • The major aim of these designs is to establish a causal connection between interventions (independent variable) and outcomes (dependent variables)
Classical Experimental Design • Considered to be the “ideal” experiment as all six requirements of are present: • Time order of IV; manipulation of IV; relationship between IV and DV; rival hypotheses controlled; control group; random selection and random assignment • Design Blueprint: R O1X O2 R O1 O2
Randomized Posttest-Only Control Group Design • Another version of the “ideal” experiment where only one observation or measure is made of the dependent variable for each group • Design Blueprint: R X O1 R O1
SUMMARY • Group-level designs are categorized according to the knowledge level continuum • Exploratory, descriptive, explanatory • Threats to internal and external validity are highest for exploratory designs and lowest for explanatory designs