What is research methods?

1. What is research methods?

2. Reviewing the ads • What sorts of methods are used in the advertisements? • Which of these is the most trustworthy method? • Why?

3. Thinking more about research methods • Designed to help prevent mistakes that people naturally make when trying to find out about the world

4. Human nature… • As part of human nature, people want to find out about the world • But people also want to feel like they’re correct • And to be able to predict and control what will happen

5. Consequences of human nature… • Illusory correlation: people want to feel like they can predict what will happen in the world •  they try to figure out what’s related to outcomes that they care about •  they see relationships that aren’t really there • E.g., lucky t-shirt, lucky pen

6. More consequences of human nature… • Confirmation bias: people want to be correct in their ideas of the world •  pay the most attention to evidence that supports their ideas •  pay little, if any, attention to evidence that doesn’t support their ideas • E.g., every time wash car it rains – only notice times when wash car and it rains

7. More consequences of human nature… • Changing definitions: people want to be right, or want to make a particular conclusion about the world •  they may strategically change their definitions • E.g., good students, good professors, good drivers

8. More consequences of human nature… • Ignoring the need for comparison groups: people tend to think that they know more than they do •  focus on what’s in front of them, and ignore other potentially relevant information • E.g., Today show: who proposes; who’s a serial killer

9. Making some decisions • A panel of psychologists has interviewed and administered personality tests to 30 engineers and 70 lawyers, all successful in their respective fields. On the basis of this information, brief descriptions of these 100 individuals have been written. You will find below the description of one of these individuals, chosen at random from the 100 available. Please indicate what you think the probability is that person described is an engineer, on a scale from 0% to 100% likely.   • Richard is a 30-year-old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues. • What is the likelihood that Richard is an engineer ________%

10. Another decision •   Richard is a 30-year-old man. He is married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues. • What is the likelihood that Richard is an engineer ________%

11. One more • Tom W. is a professor at a university. Tom is of high intelligence, although he lacks true creativity. He has a need for order and clarity, and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, but it is clear and concise. Outside of the classroom Tom is quiet and introverted, likes to work on analytical reasoning puzzles, and wears rather unfashionable "nerdy" clothes.        • Tom is a professor in either a Humanities department (such as English or History) or the Statistics department. • What is the likelihood that Tom is a professor of Humanities?  ________%

12. A last decision • Bill Z. is a professor at a university. He is a member of either a Humanities department (such as English or History) or the Statistics department.        • What is the likelihood that Bill is a professor of Statistics?  ________%

13. Putting it all in perspective • What did you say for Richard, and whether he was an engineer or a lawyer? • What made sense, given the information provided? • 70% of the descriptions were of lawyers, and this one was drawn at random • How might illusory correlation have played a role here?

14. Thinking about the other decisions • What did you say for the second description, about Tom W? • Based on the information given, what should you have said? • He was either a Humanities or Statistics professor  50-50 shot that he’s one or the other • How might illusory correlation have played a role? • What did you say for the last description, of Bill Z? • Why was it easier to rely on the 50-50 for Bill than for Tom?

15. More illusory correlation and such • Illusory correlation slides… • IllusoryCorrelationDemonstration.ppt

16. Challenge of research methods • Need to overcome each of these • Need way to make sure that people pay attention to all the relevant information, keep the same definition, don’t see patterns that aren’t there, and don’t overestimate what they know

17. The mechanics of research methods • Involves specifying a definition • Involves giving equal weight to all evidence •  systematic gathering of evidence

18. Bringing back in hypotheses • Start with research hypothesis • Which is? • Really test null hypothesis • Which is? • Collect data, and figure out likelihood of getting those data, if the null hypothesis were true • Which is? • Compare to critical cut-off likelihood • Which is? • If likelihood of getting your data is less than critical cut-off likelihood, decide…

19. Why we need the null • Statistics point of view: • Know what the world looks like if the null was true • Have that sampling distribution of the means • Can figure out probability of finding different sample means or different outcomes if the null was true

20. More on why it’s the null that’s tested • Remember, when we study people, we can’t study the entire population we care about • Instead, we need to just use information from a sample of people • Let’s say, for instance, that we’re studying swans, and are testing the idea that all of them are white • We find a sample of 100 white swans… does this mean that *all* swans are white?

21. More on why it’s the null that’s tested

22. What do the swans mean? • When we only have access to data from a sample of people from the population that we care about, we can use that sample to show that something is *not* true (e.g., by finding one black swan), but we can’t use it to show that something *is* true for the whole population (e.g., that every single swan is white)

23. Another way to think of things… • Think back to logic from high school… • p  q • Does that mean that, if q is true, p is true? • In words: • If all swans are white, then this sample of swans will be white • But… if this sample of swans is white, does it mean that *all* swans are white? • Or… • If you love research methods, you’ll show up to class • But… if you show up to class, does this mean you love research methods?

24. Bringing in fancy philosophy words • You cannot affirm the consequent • You can’t say that, just because q is supported, that means that p is true

25. Avoiding the word “prove” • As a consequence of this, the word “prove” should never appear in a research paper • The research hypothesis is not proven correct • The null hypothesis is not proven incorrect – or correct • Why not? • Just because something is true in a sample does not mean it is true in a population • Remember Type 1 and Type 2 errors

26. So, one ingredient in research methods • Is hypotheses (and null hypotheses)

27. Making sense of all those results • Put them together to create a theory • Theory = more general than hypothesis • Combines together results of many individual studies • Can also be used to generate new hypotheses for new research • Over time, theories get refined as more research is conducted

28. How do we get the data to test these hypotheses? • Moving from theory to hypothesis, and hypothesis to methods of testing that hypothesis is the key to research methods • Research methods = mechanics of how to systematically test predictions about the world

29. Findings when research methods are used… • More trustworthy than findings when other methods (like testimonial, method of authority, etc) are used • Why?

30. Other considerations • Ethics… dealing with collecting data from humans (or other animals)  need to treat them in an ethical manner