Basics of Experimentation (1) Experimental Design: Which to Choose and Why?

1. Basics of Experimentation (1)Experimental Design: Which to Choose and Why?

2. Why Conduct on Experiment? • Goal of experimentation: • To determine cause-and-effect relations in nature • “Factor A causes factor B” • What constitutes a good experiment? • Validity of the results: how true the results are • Internal Validity • Extent to which one can make causal statements about the relationship between variables • Experiments aim to maximize their internal validity

3. Variables • Variable • An event or behavior that can assume two or more values • Ex: temperature; speed of stimuli presentation; medication • Independent Variable (IV) • Selected and manipulated by the experimenter • Levels: must have at least 2 levels for comparison • Ex: Cell phone use: no cell phone, hand held phone, hands free phone • Dependent Variable (DV) • Response or behavior that is measured by the experimenter • Ex: reaction time to pressing the brake pedal; # of words recalled; symptoms of anxiety

4. Experimental Designs • How will participants be assigned to the levels of the IV? • Between-Subjects Design • Different sets of participants are assigned to only 1 level of the IV • Within-Subjects Design • Each participant is assigned to all levels of the IV Level 2 Jack Sam Lisa Level 1 Bill John Sally Level 3 Eric Peter Sandy Level 3 Larry Bob Jenny Level 2 Larry Bob Jenny Level 1 Larry Bob Jenny

5. Example • The effects of cell phone use while driving on the time for the driver to realize the car in front of him/her is stopping. • Independent variable • cell phone use • 3 levels: no cell phone, hand held phone, hands free phone • Dependent variable • Time it takes participants to press the brake when the car in front of them brakes.

6. Between-Subjects Design • Considered “safer” • No chances of contamination from one treatment condition to the other. • The same participant never gets more than one treatment. • Experimenter must check for as few differences as possible to exist between participants across all treatment conditions before testing. • Reduces chances of confounded experiment • Maximizes cause-and-effect relations

7. Between-Subjects Design No Cell Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Hands Free Phone Condition Participant 13 Participant 14 Participant 15 Participant 16 Participant 17 Participant 18 Hand Held Phone Condition Participant 7 Participant 8 Participant 9 Participant 10 Participant 11 Participant 12 Total of 18 participants needed in this experiment

8. Two ways to increase internal validity • 1- Matching • Select a variable(s) other than the IV that could mostly likely affect performance on DV. • Ex: vision, hours of video game playing, reaction time to pressing a pedal, age • Create triads of participants who are equal on all these variable. • Randomly assign each member of the triads to a level of IV. • Problems: • Cannot match participants for everything • Difficult to know which is the most important variable to match

9. Two ways to increase internal validity • 2- Randomization • Each participants has an equal chance of being in any of the conditions of the experiment. • Does not guarantee groups will always be equal. • Preferred method when basis of matching is unsure.

10. Advantages Simpler to conduct. No chances of contamination across treatment conditions. Disadvantages Requires more participants. Results may be confounded if groups are not equated by randomization. Difficult to determine which factors are important to match participants. Between-Subjects Design

11. Within-Subjects Design • All subjects receive all levels of the IV • The performance of each subject is compared across the different experimental conditions. • Reduces the effects of individual differences across conditions. • Carryover Effects • Performance on one condition affects performance on a follow-up condition(s). • Reduces internal validity of experiment.

12. Within-Subjects Design No Cell Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Hands Free Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Hand Held Phone Condition Participant 1 Participant 2 Participant 3 Participant 4 Participant 5 Participant 6 Total of 6 participants are needed in this experiment

13. To reduce carryover effects Counterbalancing • Technique used to counter the effects of presenting conditions in a particular order/sequence. Requirements: • Each condition must the presented to each participant an equal number of times. • Each condition must occur an equal number of times in each treatment order. • Each condition must precede and follow each of the other treatments and equal number of times.

14. How to know the number of sequences • How do you know the number of sequences that you need to run to meet these requirements? • Formula: n! (n factorial) • n = number of conditions; n = 3 • 3! = 3 x 2 x 1 = 6 sequences • 5 conditions? 6 conditions? • As the number of conditions increase, the number of sequences becomes too large. • Alternative procedure: Balanced Latin Square

15. Balanced Latin Square • Each condition precedes and follows every other condition an equal number of times. • Preferred method of counterbalancing • For even number of conditions • first row formula = 1, 2, n, 3, n -1, 4, n - 2… Need to run participants in multiples of n conditions, or 4

16. Odd number of conditions - uses 2 Latin Squares • The 2nd Latin square is the mirror image of the 1st. • Need to run subjects in multiples of 2n, or 2 x3 = 6.

17. Odd number of conditions - uses 2 Latin Squares

18. Advantages Requires fewer participants. Reduces individual differences. Disadvantages Potential for carryover effects if conditions are not counterbalanced. Within-Subjects Design

19. Which Design Works Best? • It depends! • If you have a large sample size and predict IV will have a large effect: • Use a between-subjects design. • If you have a limited sample size and predict IV will have a small chance of carryover effects: • Use a within-subjects design.

20. Cross-sectional designs • Study individuals of different ages at the same time. • Ex: effect of age on cognitive abilities (memory) • Test 7- yr olds, 25 yr olds and 60 yr olds • Advantage • A wide variety of ages can be studied at the same time • Can collect data quickly, maybe all at once. • Disadvantage • Individuals are born at different times and raised with different generations; different educational systems • Cohort effect: era in which individuals are born affects how they respond in a study.

21. Longitudinal Design • Same participants are studied repeatedly over time as they age. • Ex: test the same participants every 3 years: 9 yr old, 12 yr old, 15 yr old… • Advantage: no cohort effect • Disadvantage • Attrition rate increases: participants drop out of study • Expensive and time consuming • Participants who remain may not be a representative sample

22. Sequential Design • Combination of longitudinal and cross-sectional designs • Test 5 yr olds, 15 yr olds and 45 yr olds ever 3 years • Advantages • Allow to examine age effects without taking as much times as a longitudinal study. • Disadvantages • Expensive and time consuming • Attrition rates