1 / 51

III. Research Design Part I: Experimental Designs

III. Research Design Part I: Experimental Designs. Hypotheses. A hypothesis is a testable statement of causal relationship between two variables , derived from theory Must specify a relationship between an independent and dependent variable

ayasha
Download Presentation

III. Research Design Part I: Experimental Designs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. III. Research Design Part I: Experimental Designs

  2. Hypotheses • A hypothesis is a testable statement of causal relationship between two variables, derived from theory • Must specify a relationship between an independent and dependent variable • Clear, specific, amenable to empirical testing, value-free (F-N & N)

  3. Research Design • “The program that guides the investigator in the process of collecting, analyzing, and interpreting observations. It is a logical model of proof that allows the researcher to draw inferences concerning causal relations among the variables under investigation” (F-N and N).

  4. Relationships (Covariation) between Variables • Relationships between variables: Two variables are related to one another (i.e. are correlated) if one or more values of one variable tend to be associated with one or more values of the other variable.

  5. Directional Relationships • Apply to cases where the values of the IV and DV are orderable (directional) variables • Positive relationship: • As one’s education level increases, the frequency of voting increases • There is a positive relationship between one’s education level and voting frequency

  6. Directional Hypotheses (cont’d) • Negative relationship: • As the number of hours of negative ads watched increases, the frequency of voting decreases • There is a negative relationship between exposure to negative advertising and one’s voting frequency

  7. Non-Directional Hypotheses • Appropriate for variables that are not orderable • Hypothesis describes comparison among categories • Examples • Men have greater levels of support for President Bush than do women • Whites are most likely to be Republican, while African-Americans are most likely to be Democrat

  8. Research Design and Causality • Relationships between variables: Two variables are related to one another (i.e. are correlated) if one or more values of one variable tend to be associated with one or more values of the other variable. • Causal relationship: A relationship in which one variable directly causes/explains the other variable.

  9. Establishing Causality (F-N&N) • 3 Criteria (Evidence Needed) for Establishing Causality • Covariation (X is correlated with Y) • Time Order (X precedes Y in time) • Nonspuriousness (The observed relationship between X and Y is not spurious)

  10. Spurious Relationship • A relationship between two variables that is presumed to be causal, when in fact it is not • An observed relationship between X and Y is said to be spurious (or partly spurious) if there exists a third variable Z (a “control variable”) which is both a cause of Y AND is correlated with X.

  11. Spurious Relationship (Presumed Causal Relationship) Y X (True Causal Relationship) Z

  12. Example of Spuriousness

  13. Gender and Corruption – Causal or Spurious? http://www.iq.harvard.edu/blog/pb/2005/10/sex_and_corruption.html

  14. The Democratic Peace: Causal or Spurious? War Democracy Z(???)

  15. Experimental Designs 1. Select a sample 2. Randomly assign subjects into 2 or more groups. The number of groups is equivalent to the number of values of the independent variable(s). 3. Observe (measure) DV for all groups (if design includes pretest) 4. Introduce the stimulus (IV) 5. Observe (measure) DV for each group 6. If the change in the value of the dependent variable varies significantly across groups, then we conclude that X  Y • Key distinguishing features of an experimental design: Randomization and Manipulation of IV by the researcher (when introduced and to whom)

  16. Experimental Designs Measure the DV Introduce the IV Measure the DV Random Assignment Treatment Group Treatment Group “Stimulus” A bunch of people Measure The DV Measure the DV Random Assignment Control Group Control Group “Placebo” (PRE-TEST) (POST-TEST)

  17. Simple Experimental Designs 2-Group Pretest - Posttest Design (Classical or “Simple” Experiment) • R O1 X O2 • R O3 O4

  18. Simple Experimental Designs 2-Group Pretest - Posttest Design (Classical or “Simple” Experiment) • R O1 X O2 • R O3 O4 • OR • R O1 X1 O2 • R O3 X2 O4

  19. Experiments and Causality • Correlation?

  20. Experiments and Causality • Covariation? • Comparison of two or more groups (on dependent variable) experiencing different levels of exposure to the causal (explanatory) variable (X). This establishes covariation.

  21. Experiments and Causality • Time Order?

  22. Experiments and Causality • Time Order? • The introduction of the independent variable (“stimulus”) is manipulated by the researcher to insure that changes in IV precede changes in DV.

  23. Experiments and Causality • Spuriousness?

  24. Experiments and Causality • Spuriousness? • Random assignment insures that rival hypotheses are ruled out, thus eliminating the threat of spuriousness. (How?)

  25. Experiments and Causality • Spuriousness? • Random assignment insures that rival hypotheses are ruled out, thus eliminating the threat of spuriousness. (How?) • Use of “matching” as a strategy to control for rival explanations

  26. Simple Experimental Designs • 2-Group Posttest Only Design • R X O1 • R O2

  27. Simple Experimental Designs • 2-Group Posttest Only Design • R X O1 • R O2 • OR • R X1 O1 • R X2 O2

  28. Other Types of Experimental Designs • Multiple Group Pretest - Posttest Design R O1 X1 O2 R O3 X2 O4 R O5 X3 O6 R O7 X4 O8 • Multiple Group Posttest Only Design R X1 O1 R X2 O2 R X3 O3 R X4 O4

  29. Other Types of Experimental Designs • Solomon 4-Group Design R O1 X O2 R O3 O4 R X O5 R O6

  30. Extensions of the Classical Experiment • Multiple observations over time R O1 X1 O2 O3…… R O4 X2 O5 O6……

  31. Extensions of the Classical Experiment • Factorial designs - each group represents a unique combination of values on two (or more) different variables. • For two independent variables X and Z (where X and Z each take on two possible values): R O1 X1, Z1 O2 R O3 X2, Z1 O4 R O5 X1, Z2 O6 R O7 X2, Z2 O8 • Factorial designs allow the researcher to test for an interaction effect: Two independent variables interact to affect a dependent variable if the effect of one variable depends on the value of the second variable.

  32. Zilber and Niven (SSQ) • Table 1: 2-Group Posttest Only R X1(black) O1 R X2(A-A) O2

  33. Zilber and Niven (SSQ) • Table 3: 2X2 Factorial Design • R (black/liberal) O1 • R (A-A/liberal) O2 • R (black/conserv) O3 • R (A-A/conserv) O4 • To see how the effect of racial label varies as a function of ideology, we compare O1-O2 to O3-O4

  34. Zilber and Niven (SSQ) • Table 3: 2X2 Factorial Design • R (black/liberal) O1 • R (A-A/liberal) O2 • R (black/conserv) O3 • R (A-A/conserv) O4 • Conclusion: The choice of racial labels does affect white attitudes toward blacks, but only among liberals.

  35. 1.What effect do A & B have on O? 2. Is there an interaction effect? (Explain)

  36. Evaluating Research Designs:Internal Validity • Internal Validity - the degree to which we can be sure that the independent variable caused the dependent variable within the current sample • “Extrinsic Factors”: Selection effects with respect to recruitment/assignment of subjects (units) to treatment and control groups • “Intrinsic Factors”: Factors threatening validity that… • occur outside the “laboratory” during the period of the study • result from changes in, reactions to (or general ineffectiveness of) the measuring instrument, or • involve some type of reactive effect of observation

  37. Evaluating Research Designs:Internal Validity • Extrinsic factors: • Selection • Important intrinsic factors include: • History • Maturation • Experimental mortality • Instrumentation • Testing • Regression artifact • Interactions with selection - e.g. “selection history” and “selection maturation”

  38. For each of the following intrinsic threats to internal validity, explain: • What the specific threat means • Whether or not (and why or why not) an experimental design is protected from this threat

  39. History? • R O1 X1 O2 • R O3 X2 O4

  40. Maturation? • R O1 X1 O2 • R O3 X2 O4

  41. Experimental Mortality? • R O1 X1 O2 • R O3 X2 O4

  42. Instrumentation? • R O1 X1 O2 • R O3 X2 O4

  43. Testing? • R O1 X1 O2 • R O3 X2 O4

  44. Regression Artifact? • R O1 X1 O2 • R O3 X2 O4

  45. “Selection History” and “Selection Maturation” • R O1 X1 O2 • R O3 X2 O4

  46. Evaluating Research Designs:External Validity • External Validity - the degree to which the results of the analysis can be generalized beyond the current sample/study. Can be maximized by: • Using subjects (units) that are representative of the population to which one’s theory applies • Using a “laboratory” that is as close to “real life” conditions as possible • Field experiments

More Related