Research Methods Safeguards against error

1. 2 Research Methods Safeguards against error Slides prepared by Matthew Isaak

2. Learning Objectives LO 2.1 Identify two modes of thinking and their application to scientific reasoning. LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. LO 2.3 Describe the role of correlational designs and distinguish correlation from causation.

3. Learning Objectives LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. LO 2.5Explain the ethical obligations of researchers toward their research participants. LO 2.6 Describe both sides of the debate on the use of animals as research subjects. LO 2.7 Identify uses of various measures of central tendency and variability.

4. Learning Objectives LO 2.8 Explain how inferential statistics can help us to determine whether we can generalize from our sample to the full population. LO 2.9Show how statistics can be misused for purposes of persuasion. LO 2.10Identify flaws in research designs and how to correct them. LO 2.11Identify skills for evaluating psychological claims in the popular media.

5. Lecture Preview • The beauty and necessity of good research design • The scientific method • Ethical issues in research design • Statistics • Evaluating psychological research

6. Why We Need Research Designs LO 2.1 Identify two modes of thinking and their application to scientific reasoning. • In the early 1990s, an autism treatment was developed called "facilitated communication." • The developers thought that autism was a motor disorder. • The facilitator sat next to child with autism and guided the child's hand over a keyboard, allowing the children to type out words.

7. Why We Need Research DesignsLO 2.1 Identify two modes of thinking and their application to scientific reasoning. • Students seemed to make stunning progress in communication, telling parents "I love you" and writing poetry. • However, some students began making allegations of sexual abuse against parents. • There was no physical evidence, just the communicators via the facilitators.

8. Why We Need Research DesignsLO 2.1 Identify two modes of thinking and their application to scientific reasoning. • Dozens of controlled studies examined the phenomenon and found that the words came solely from the minds of the facilitators. • Still, some people continue to practice facilitated communication.

9. Figure 2.1 Putting Facilitated Communication to the Test.

10. Why We Need Research DesignsLO 2.1 Identify two modes of thinking and their application to scientific reasoning. • Even well-educated, intelligent people can be fooled. • Well-planned designs can help to eliminate biases when examining phenomena.

11. Why We Need Research DesignsLO 2.1 Identify two modes of thinking and their application to scientific reasoning. • Prefrontal lobotomy is example of what happens when we rely on subjective impressions. • Developer won the Nobel Prize • In it, the neural fibers connecting frontal lobes to the thalamus were severed. • Control studies showed it didn't work.

12. Figure 2.2 The Prefrontal Lobotomy.

13. Two Modes of ThinkingLO 2.1 Identify two modes of thinking and their application to scientific reasoning. System I or intuitive thinking • Quick, reflexive, almost automatic • Relies on heuristics System 2 or analytical thinking • Slow, reflexive, effortful

14. Two Modes of ThinkingLO 2.1 Identify two modes of thinking and their application to scientific reasoning. • Heuristics are mental shortcuts or rules of thumb that we use daily. • They reduce the cognitive energy required to solve problems but we oversimplify reality. • Imagine yourself driving from Reno, Nevada to San Diego, California—which compass direction would you take?

15. San Diego is actually EAST of RenoLO 2.1 Identify two modes of thinking and their application to scientific reasoning. FIGURE 2.3 In Which Compass Direction Would You Travel to Get from Reno, NV to San Diego, CA?

16. So, how do we prevent ourselves from being fooled by our own (and other people's) biases? LO 2.1 Identify two modes of thinking and their application to scientific reasoning.

17. The Scientific Method Toolbox LO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • Allows us to test specific hypotheses derived from broader theories of how things work. • Theories are never "proven," but hypotheses can be confirmed or disconfirmed. • We can use a number of different types of SM tools to gain information and test hypotheses.

18. Naturalistic ObservationLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • Watching behavior in real-world settings • High degree of external validity - extent to which we can generalize our findings to the real world • Low degree of internal validity - extent to which we can draw cause-and-effect inferences

19. Case Study DesignsLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • Studying one person or a small number of people for an extended period of time • Common with rare types of brain damage or mental illness • Helpful in providing existence proofs, but can be misleading and anecdotal

20. Self Report Measures and SurveysLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • Psychologists often need to ask people about themselves or others. • Self-report measures or questionnaires asses characteristics such as personality or mental illness. • Surveys ask about a person's opinions or abilities. • Not all measures and surveys are equal.

21. Random SelectionLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • The key to generalizability in surveys and questionnaire studies • Ensures every person in a population has an equal chance of being chosen to participate • Non-random selection can skew results and make them inaccurate when applied to the population as a whole.

22. Evaluating MeasuresLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • To trust results, the measures must have: • Reliability—consistency of measurement • Validity—extent to which a measure assesses what it claims to measure • A test must be reliable to be valid, but a reliable test can still be completely invalid.

23. Self-Report MeasuresLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • Pros • Easy to administer • Direct (self) assessment of person's state

24. Self-Report MeasuresLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • Cons • Accuracy is skewed for certain groups (narcissists) • Potential for dishonesty • Response sets - tendencies of research subjects to distort their responses • Positive impression management • Malingering

25. Rating DataLO 2.2 Describe the advantages and disadvantages of using naturalistic observation, case studies, self-report measures, and surveys. • People can also be asked to rate others on different characteristics. • This can do away with some biases in self-report, but still has problems. • Halo effect—tendency of ratings of one positive characteristic to spill over to influence the ratings of other characteristics

26. Correlational Designs LO 2.3 Describe the role of correlational designs and distinguish correlation from causation. • Examine how two variables are related • Correlations vary from -1 to +1 and can be: • Positive (as one increases, so does the other) • Negative (as one increases, the other decreases) • Zero (no relationship between variables) • Depicted in a scatterplot

27. ScatterplotsLO 2.3 Describe the role of correlational designs and distinguish correlation from causation. Figure 2.4 Diagram of Three Scatterplots.

28. Correlational DesignsLO 2.3 Describe the role of correlational designs and distinguish correlation from causation. • Illusory Correlation—perception of a statistical association where none exists • Crime rates and the full moon • Arthritis and weather • Examining a probability table helps to explain why we are all prone to seeing relationships where none exists.

29. The Great Fourfold Table of LifeLO 2.3 Describe the role of correlational designs and distinguish correlation from causation. Did a crime occur? Yes No Yes Did a full moon occur? No Humans tend to overemphasize cell A and ignore the non-events.

30. Correlation vs. CausationLO 2.3 Describe the role of correlational designs and distinguish correlation from causation. • Just because two things are related, does not mean that one causes another. • There are three possible explanations: • A causes B • B causes A • C causes both A and B

31. Determining Causation LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • The only way to determine if one thing is casually related to another is via an experimental design. • This is because in an experiment, you purposefully manipulate variables, rather than just measure already existing differences.

32. What Makes a Study an Experiment?LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • Random assignment of participants groups • Experimental Group - receives the manipulation • Control Group - does not receive the manipulation

33. What Makes a Study an Experiment?LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • Manipulation of an independent variable • The dependent variable is what the experimenter measures to see whether manipulation had an effect.

34. What Makes a Study an Experiment?LO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • Confounds - any difference between the experimental and control groups aside from IV • Makes IV effects uninterpretable • Cause and effect - possible to infer, with random assignment and manipulation of independent variable

35. Pitfalls in Experimental DesignLO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • Placebo effect - improvement resulting from the mere expectation of improvement • Participants must be blind to their assignment to groups. • Placebos show many of the same characteristics as real drugs. • Nocebo effect - harm resulting from the mere expectation of harm

36. Pitfalls of Experimental DesignLO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • Experimenter expectancy effect – when researchers' hypotheses lead them to unintentionally bias a study outcome • Clever Hans, the mathematical horse • Rosenthal's undergrads and maze-bright or maze-dull rats • Using a double-blind design can decrease this.

37. Pitfalls of Experimental DesignLO 2.4 Identify the components of an experiment, the potential pitfalls that can lead to faulty conclusions, and how psychologists control for these pitfalls. • Demand characteristics - cues that participants pick up allowing them to guess at the researcher's hypotheses • Disguising the purpose of the study or using "filler" items can help to decrease these.

38. Ethical Issues in Research Design LO 2.5 Explain the ethical obligations of researchers toward their research participants. • Tuskegee study ran from 1932 to 1972 • African American men living in rural Alabama diagnosed with syphilis • U.S. Public Health Service never informed, or treated, the men; they merely studied the course of the disease. • 28 men died of syphilis, 100 of related complications, 40 wives were infected, and 19 children were born with it.

39. Ethical Guidelines for Human ResearchLO 2.5 Explain the ethical obligations of researchers toward their research participants. • Today, research has to go through a careful process of review to ensure that it is conducted ethically. • Institutional Review Board (IRB) • Informed consent • Justification of deception • Debriefing of subjects afterwards

40. Ethical Issues in Animal Research LO 2.6 Describe both sides of the debate on the use of animals as research subjects. • Only 7-8% of psychological research uses animals. • Vast majority are rodents and birds • Goal is to generate ideas about the brain and behavior without harming people

41. Statistics: The Language of Research LO 2.7 Identify uses of various measures of central tendency and variability. • Descriptive statistics—numerical characterizations of the data set • Central tendency—where the group tends to cluster • Mean: average of all scores • Median: middle score in the data set • Mode: most frequent score in the data set

42. Figure 2.7 Distribution Curves.

43. Statistics: The Language of ResearchLO 2.7 Identify uses of various measures of central tendency and variability. • Variability—sense of how loosely or tightly bunched scores are • Range—difference between the highest and lowest scores • Standard deviation—measure that takes into account how far each data point is from the mean

44. Range vs. Standard DeviationLO 2.7 Identify uses of various measures of central tendency and variability. • Both sets of data have the same range, but very different standard deviations. • Standard deviations are less susceptible to extreme scores than ranges are.

45. FIGURE 2.8 The Range versus the Standard Deviation.

46. Statistics: The Language of Research LO 2.8 Explain how inferential statistics can help us to determine whether we can generalize from our sample to the full population. • Inferential statistics allow us to determine whether we can generalize findings from our sample to the population. • Statistical significance - finding would have occurred by chance less than 1 in 20 times • Practical significance - real-world importance

47. How People Lie With Statistics LO 2.9 Show how statistics can be misused for purposes of persuasion. • People can misuse statistics to persuade – and mislead – others. They can: • Report unrepresentative measures, like the mean instead of the median for skewed data • Truncate the axes of graphs • Neglect base rate probabilities

48. Figure 2.9 Arrest Rates Before and After Transcendental Meditation.

49. Evaluating Psychological Research LO 2.10 Identify flaws in research designs and how to correct them. • The process of peer review helps to identify and correct flaws in research and research conclusions. • Remember to keep a look out for confounds, placebos, experimenter expectancy, correlation vs. causation, and others.

50. Evaluating Psychology in the Media LO 2.11 Identify skills for evaluating psychological claims in the popular media. • Most reporters are not scientists, so consider the source. • Tabloids vs. science magazines • Beware of sharpening, leveling, and pseudosymmetry.