Experimental Research Designs

1. Experimental Research Designs

2. Difficult to establish cause-and-effect. Correlational research often done first to establish relationships that may be examined for cause-and-effect. Cause-and-effect are not established by statistics but rather by logical thinking and sound research design. You must establish that no other plausible explanation exists for the changes in the DV except the manipulation done to the IV.Difficult to establish cause-and-effect. Correlational research often done first to establish relationships that may be examined for cause-and-effect. Cause-and-effect are not established by statistics but rather by logical thinking and sound research design. You must establish that no other plausible explanation exists for the changes in the DV except the manipulation done to the IV.

3. Causality Temporal precedence Covariation between IV and DV Eliminate alternative explanations

4. Types of Experimental Designs Simple True Experimental Complex True Experimental Quasi-Experimental

5. Types of Experimental Designs Simple True Experimental Complex True Experimental Quasi-Experimental

6. Simple True Experimental Characteristics Types Variations

7. Characteristics of True Designs Manipulation (treatment) Randomization Control group Characteristics of simple true designs One IV with 2 levels (T, C) One DV

8. Types Randomized posttest control group design Randomized pretest-posttest control group design Random groups, controls for past history, maturation, testing, and sources of invalidity based on nonequivalent groups (statistical regression, selection biases, selection-maturation interaction Investigator must control present history, instrumentation, experimental mortality Random groups, controls for past history, maturation, testing, and sources of invalidity based on nonequivalent groups (statistical regression, selection biases, selection-maturation interaction Investigator must control present history, instrumentation, experimental mortality

9. Randomized posttest control group design R T Post R C Post

10. Randomized pretest-posttest control group design R Pre T Post R Pre C Post Makes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample sizeMakes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample size

11. Advantages & Disadvantages Advantages of pretest design Equivalency of groups Can measure extent of change Determine inclusion Assess reasons for and effects of mortality Disadvantages of pretest design Time-consuming Sensitization to pre-test

12. Solomon four-group design R Pre T Post R Pre C Post R T Post R C Post

13. Variations Independent groups (between groups) Repeated measures (within groups)

14. Repeated Measures Design Advantages: Fewer subjects needed (less costly) Sensitive to finding statistical differences Disadvantages: Order effect (practice, fatigue, carry-over) Advantages: Fewer subjects needed (less costly) Sensitive to finding statistical differences because of control over participant differences Disadvantages: Order effect (practice, fatigue, carry-over) Advantages: Fewer subjects needed (less costly) Sensitive to finding statistical differences because of control over participant differences Disadvantages: Order effect (practice, fatigue, carry-over)

15. Dealing with Order Effects Counterbalancing n! Latin squares Counterbalancing - all possible orders of presentation are included in experiment Latin squares � a limited set of orders constructed to ensure that Each condition appears at each ordinal position Each condition precedes and follows each condition one timeCounterbalancing - all possible orders of presentation are included in experiment Latin squares � a limited set of orders constructed to ensure that Each condition appears at each ordinal position Each condition precedes and follows each condition one time

16. Latin Squares

17. Dealing with Order Effects Counterbalancing n! Latin squares Randomized blocks Time interval between treatments Time interval � may counteract the effects of treatment, but also increases time demands on subjectsTime interval � may counteract the effects of treatment, but also increases time demands on subjects

18. Variations Independent groups (between) vs. repeated measures (within) designs

19. Types of Experimental Designs Simple True Experimental Complex True Experimental Quasi-Experimental Pre-Experimental

20. Characteristics of True Designs Manipulation (treatment) Randomization Control group Characteristics of simple true designs One IV with 2 levels (T, C) One DV

21. Complex True Experimental Randomized matched control group design Increased levels of IV Factorial design Multiple DVs

22. Complex True Experimental Randomized matched control group design Increased levels of IV Factorial design Multiple DVs

23. Randomized matched control group design M R T Post M R C Post Obtain measure of matching variable from each subject Rank from highest to lowest based on score Form matched pairs Randomly assign members of pairs to conditionsObtain measure of matching variable from each subject Rank from highest to lowest based on score Form matched pairs Randomly assign members of pairs to conditions

24. Complex True Experimental Randomized matched control group design Increased levels of IV Factorial design Multiple DVs

25. Increased Levels of IV Provides more complete information about the relationship between the IV & DV Detects curvilinear relationships Examines effects of multiple treatments

27. Increased Levels of IV

28. Complex True Experimental Randomized matched control group design Increased levels of IV Factorial design Multiple DVs

29. Factorial Design >1 IV (factor) Simultaneously determine effects of 2 or more factors on the DV (real world) Between Factor vs. Within Factor ID�d by # of factors and levels of factors Use Figures 10.1 and 10.2 to demonstrate graphically at end of slide.Use Figures 10.1 and 10.2 to demonstrate graphically at end of slide.

30. Do differing exercise regimens (hi, med, lo intensity) have the same effect on men as they do on women? 3 X 2 (Exercise Regimen X Gender) 2 factors Exercise Regimen � 3 levels Gender � 2 levels Between factors DV? Experimental IVs or Participant IVs?

32. Do strength gains occur at the same rate in men as they do in women over a 6 mo. training period? Measurements are taken at 0, 2, 4, 6 mo. 2 X 4 (Gender X Time) ? factors Time � 4 levels Gender � 2 levels Between or within factors? DV? Experimental IVs or Participant IVs?

34. Cell means, Margin meansMain Effects, Interactions

35. Interaction & Main Effects Interaction - combined effect of the factors on the dependent variable Main effect � the deviation of two or more treatment means from the grand mean Significant interaction means that the effect that one factor has on the dependent variable depends on which level of the other factor is being administered Significant interaction means that the effect that one factor has on the dependent variable depends on which level of the other factor is being administered

37. Parallel lines indicate no interaction.

39. Non-parallel lines indicate an interaction.

41. Interpretation Always interpret the interaction first (graphical) If no significant interaction, interpret main effects

42. Advantages of factorial designs: Greater protection against Type I error More efficient Can examine the interaction Disadvantages: ? subject # for between factor designs Advantages of factorial designs: Greater protection against Type I error More efficient (1 analysis vs. multiple one-ways) Can examine the interaction (not possible with one-way ANOVAs) Disadvantages: Only for fixed models (levels of IV chosen by researcher) and when subjects are assigned randomly Advantages of factorial designs: Greater protection against Type I error More efficient (1 analysis vs. multiple one-ways) Can examine the interaction (not possible with one-way ANOVAs) Disadvantages: Only for fixed models (levels of IV chosen by researcher) and when subjects are assigned randomly

43. IV A: Exposure to Violence � violent vs. nonviolent videoIV B: Gender � male vs. femaleDV: # ads recalled (0-8)

44. IV A: Exposure to Violence � violent vs. nonviolent videoIV B: Gender � male vs. femaleDV: # ads recalled (0-8)

45. Complex True Experimental Randomized matched control group design Increased levels of IV Factorial design Multiple DVs

47. Types of Experimental Designs Simple True Experimental Complex True Experimental Quasi-Experimental

48. Characteristics of True Designs Manipulation (treatment) Randomization Control group Less control More real-world Program evaluation

49. Randomized posttest control group design R T Post R C Post

50. Randomized pretest-posttest control group design R Pre T Post R Pre C Post Makes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample sizeMakes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample size

51. Quasi-experimental Designs One group posttest-only design One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

52. Quasi-experimental Designs One group posttest-only design One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

53. Randomized posttest control group design R T Post R C Post

54. One group posttest-only design (One shot study) T Post

55. Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

56. Randomized pretest-posttest control group design R Pre T Post R Pre C Post Makes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample sizeMakes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample size

57. One group pretest-posttest design Pre T Post

58. Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

59. Randomized posttest control group design R T Post R C Post

60. Non-equivalent control group design (Static group comparison design) T Post C Post

61. Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

62. Randomized pretest-posttest control group design R Pre T Post R Pre C Post Makes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample sizeMakes it possible to ascertain that groups were equivalent at the beginning of the study. Not necessary if Randomization was used Large sample size

63. Non-equivalent control group pretest-posttest design Pre T Post Pre C Post

64. Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

65. Time series Pre Pre Pre Pre T Post Post Post Post

66. Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

67. Quasi-experimental Designs One shot study One group pretest-posttest design Non-equivalent control group design Non-equivalent control group pretest-posttest design Time series Single subject designs (Case study) Developmental designs

68. Developmental Research Designs Longitudinal Powerful (within subject) Time consuming Attrition Testing effect Cross Sectional Less time consuming Cohorts problem Developmental research � studies the ways that individuals change as a function of age; age is the independent variable Longitudinal (similar to repeated measures) Powerful (within subject) but several problems Time consuming Attrition due to move, death, school rezoning may change sample characteristics (e.g., more obese subjects die, leaving non-obese subjects in sample � knowledge about obesity is not changing but rather sample is changing) Subjects become familiar with test items (learning effect or items may cause change in behavior) Cross-sectional (similar to independent groups) Less time consuming, but problems Cohorts � a group of people born at about the same time, exposed to same events in society, and influenced by same demographic trends such as divorce rates and family size. Are all age-groups really from same population? Are environmental circumstances that affect jumping performance the same today for 6 yr olds as they were when the 10 yr olds were 6? Developmental research � studies the ways that individuals change as a function of age; age is the independent variable Longitudinal (similar to repeated measures) Powerful (within subject) but several problems Time consuming Attrition due to move, death, school rezoning may change sample characteristics (e.g., more obese subjects die, leaving non-obese subjects in sample � knowledge about obesity is not changing but rather sample is changing) Subjects become familiar with test items (learning effect or items may cause change in behavior) Cross-sectional (similar to independent groups) Less time consuming, but problems Cohorts � a group of people born at about the same time, exposed to same events in society, and influenced by same demographic trends such as divorce rates and family size. Are all age-groups really from same population? Are environmental circumstances that affect jumping performance the same today for 6 yr olds as they were when the 10 yr olds were 6?

69. Choosing a Research Design Best addresses the problem Ethics Cost in time and money Validity (internal & external)