Reasoning in Psychology Using Statistics

1. Reasoning in Psychology Using Statistics Psychology 138 2019

2. Don’t forget Exam 1 is coming up (Feb 6) • In class part – multiple choice, closed book • In labs part – open book/notes • Today • Nuts and bolts of experimental & observational designs Announcements

3. Generally the process involves a number of steps: • identification of your research questions • identifying your variables • specify your hypotheses (how are the variables related to one another) • selecting a research design • collecting your data, analyzing your data • drawing conclusions from your data about your hypotheses Designing your research

4. Today’s focus • Generally the process involves a number of steps: • identification of your research questions • identifying your variables • specify your hypotheses (how are the variables related to one another) • selecting a research design • collecting your data, analyzing your data • drawing conclusions from your data about your hypotheses Designing your research

5. Context • Research design • Measurement scale • What is known about the distributions Statistical tests • Which tests to conduct • How to set up your datafile Design Decision Tree

6. Issue: What is the best way to study for a test? Cram the night before - or - Spread your studying out over several nights • How could we investigate this question? • What research methods should we use? Selecting your research method

7. Observational Experimental • Issue: What is the best way to study for a test? • involve examining variables as they already are • involve manipulation of variables • How could we investigate this question? • What research methods should we use? Selecting your research method

8. Randomly select individuals Watch & classify their study habits See how they do on a test Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test • Issue: What is the best way to study for a test? Observational Experimental Selecting your research method

9. Randomly select individuals Watch & classify their study habits See how they do on a test • Issue: What is the best way to study for a test? • Here our analysis looks for a relationship between the explanatory (study method) and response (test performance) variables Observational • The scale of measurement used for the variables is important Observational designs

10. Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test • Why random assignment? • To try to minimize bias, potential systematic differences between the people in the different groups Experimental Part 1 of the movie “Every Statistical test starts with an appropriate selection of subjects” Experiments

11. Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test • Why 2 groups? • All experiments make some sort of comparison. Experimental Experiments

12. Randomly select individuals Tell 10 students to distribute their studying See how they do on a test Experiment ?? ?? If only one group: • This is an observational design - called “one-shot case study design” • Why 2 groups? • All experiments make some sort of comparison. • There is natural variability in your DV. • With no comparison group, so hard to see if: change in the IV change in the DV Causes • Should not make causal claims Special case: when our “comparison” group is the population Experiments

13. Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Experiment If two groups: • Why 2 groups? • All experiments make some sort of comparison. The systematic reduction and control of variability of the IV(s) allows us to examine whether they are sources of (causes) variability in the DV • Still some natural variability in your DV. • But have constrained one source of the variability (crammed or distributed studying) • Can see if there is an effect of the manipulation change in the IV change in the DV Causes We will discuss these designs with two groups later in the semester Experiments

14. Experiment • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test Experiments

15. Experiment – These are the variables that are manipulated by the experimenter • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test • A number of ways to manipulate your IV • Event/Stimulus manipulations – manipulate characteristics of the stimuli, context, etc. • Instructional manipulations – different groups are given different instructions • Subject manipulations – there are (pre-existing mostly) differences between the subjects in the different conditions (typically results in quasi-experimental designs) • Will typically use a Categorical Scale • Nominal or Ordinal – names corresponding to different levels of the IV Experiments

16. Experiment 1 IV (factor): study type 2 levels: Crammed study Distributed study Study all in one night Study over five nights • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test Experiments

17. Experiment • These are the variables that are measured by the experimenter, they are “dependent” on the independent variables • For most of our designs these will be measured on a Continuous Scale of Measurement (Ratio or Interval) Test performance • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test Experiments

18. Experiment – Holding things constant • Controls for excessive random variability Distributed study Crammed study Study for 5 hours Study for 1 hour each night Total study time = 5 hours • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test Control Variable: total time studying Experiments

19. Experiment - May freely vary, to spread variability equally across all experimental conditions • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test Random variables: How much sleep they get What they eat for lunch Experiments

20. Experiment – Other variables, that haven’t been accounted for (manipulated, measured, randomized, controlled) that co-vary with the IV(s) and can impact changes in the dependent variable(s) • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound • Randomly select individuals • Randomly assign to groups • Crammed study group • Distributed study group • See how they do on a test Experiments

21. Randomly select individuals Randomly assign to groups Crammed study group Distributed study group See how they do on a test Experiment Underlying third variable that varies with the IV Distributed study Crammed study Is this difference due to study method? Test performance • Issue: What is the best way to study for a test? • Variables • Independent • Dependent • Extraneous • Control • Random • Confound Possible Confound variable: Order of studied material, always start with vocab Studied vocabulary and practice problems Studied only vocabulary Or is it due to what was studied? 85% 73% Experiments

22. Which research method is best? - it depends Observational Experimental - May see patterns of complex behaviors - Good first step - May learn about something unexpected - Precise control possible - Precise measurement possible - Can make causal claims Advantages - Shouldn’t make causal claims - Directionality of the relationship isn’t known - Threats to internal validity due to lack of control - Sometimes the results are not reproducible - Artificial situations may restrict generalization to “real world” - Complex behaviors may be difficult to measure Disadvantages Selecting your research method

23. Your design largely determines your statistical analysis Design Decision Tree

24. Your design largely determines your statistical analysis Which test do I use? • Students often find this one of the hardest parts of the course – the key is understanding the context Design Decision Tree SPSS

25. Today’s lab Practice developing and evaluating experiments Questions? Statistics Learning CentreChoosing which statistical test to use (~10 mins) Erich Goldstein Choosing a Statistical Test (~13 mins) RStatsInstituteIndependent vs. Paired t Tests Setup in SPSS (~4 mins) Dr. Todd Grande Comparison of Independent-Samples T Test and Paired-Samples T Test using SPSS (~20 mins) Wrap up