Cognitive Biases 2

1. Cognitive Biases 2 Incomplete and Unrepresentative Data

2. “I know horoscopes can predict the future, because I’ve seen it happen.” “Positive thinking can cure cancer, because I know someone who used it, and got better.” “Everyone knows you do worse in your second year in college, you see it all the time.”

3. Necessary and Sufficient Evidence If claims like these are true, for example, if it’s true that horoscopes predict the future, then the evidence in question is necessary. If horoscopes predict the future, then there must be cases where a horoscope predicted something, and then it happened.

4. Necessary and Sufficient Evidence But such evidence is not sufficient for establishing the truth of these claims. When a horoscope predicts that X will happen, and then X happens, that doesn’t prove anything. Were there times horoscopes predicted things that didn’t happen? Were there things that horoscopes should have predicted, but never happened?

5. Contingency Tables

6. Contingency Tables A contingency table is used to plot predictions of the form “If A happens, then B will happen.” In the upper left corner of the table (Prediction = Yes, Observation = Yes) are cases where A happens and B happens. These are “true positives” and they confirm the prediction.

7. Contingency Tables A contingency table is used to plot predictions of the form “If A happens, then B will happen.” In the lower right corner (Prediction = No, Observation = No) are cases where A does not happen, and B does not happen. These are “true negatives” and also confirm the claim in question.

8. Contingency Tables A contingency table is used to plot predictions of the form “If A happens, then B will happen.” Two sorts of cases disconfirm the prediction. First there are false positives in the upper right corner, where the predicted outcome (Prediction = Yes) does not match the observed outcome (Observation = No).

9. Contingency Tables A contingency table is used to plot predictions of the form “If A happens, then B will happen.” Second, there are “false negative” cases. These are also outcomes where the prediction (Prediction = No) fails to match the observation (Observation = Yes).

10. Perfect Correlation Claims For “absolute” claims (If A happens, B will always happen, A and B are perfectly positively correlated), all cases must fall in the upper left or lower right corners. The prediction must always match the outcome.

11. Imperfect Correlation Claims For probabilistic claims (A and B are imperfectly positively correlated), you must compare the proportion of true positives to predicted positives to the proportion of false positives to predicted negatives. (How you should compare them depends on the base rates…)

12. Prediction = Yes & Observation = Yes ÷ Prediction = Yes Compared to: Prediction = No & Observation = Yes ÷ Prediction = No

13. For example… Suppose I claim that you’re more likely to get an A on the final if you take notes in class than if you take no notes.

14. Taking Notes In order to assess that claim for truth, we need to consider the ratio of students who take notes & get an A to the students who take notes: (Take notes & get an A)÷Take notes Suppose that 50% of note takers get A’s and 50% get other grades. Does that mean note takers do better or worse than non-note-takers?

15. Not Taking Notes To figure that out we need to know what proportion of non-not-takers got A’s: (No notes & got an A) ÷ No notes If 60% of the people who took no notes got A’s and 40% got other grades, then it’s false that you’re more likely to get an A if you take notes.

16. Confirmation Bias Even though evaluating predictions requires looking at both the rates of true positives among predicted positives, and the rates of false positives among predicted negatives, human beings have a tendency to only consider true positives (and to a lesser extent, true negatives) when evaluating predictions or other claims of imperfect correlation.

17. Wason Selection Task

18. Around ½ of people studied say “D” and “5”. About 1/3 say just “D”. Only about 1/20 get the right answer: “D” and “2”!

19. Searching for Confirmation People have a preference for positive answers that confirm their theories, even though negative answers that disconfirm their theories might give the same amount of information.

20. For example, suppose A picks a number between one and ten, and you’re supposed to guess what it is. I suggest that the number is 3. The psychological research shows that you’d be more likely to ask “is the number odd?” than “is the number even?”– even though both answers are equally informative.

21. In this case, the preference for confirmation does not matter: Asking the positive question and the negative question give you the same information. If people prefer the positive question, that doesn’t harm them at all. But the preference can harm them if the positive question gives less info.

22. A Strange Example Americans were asked: Which of these pairs of countries are more similar to one another? • West Germany, East Germany • Sri Lanka, Nepal

23. They said (1), West Germany and East Germany. Others (Americans) were also asked: Which of these pairs of countries are more different from one another? • West Germany, East Germany • Sri Lanka, Nepal

24. They also said (1). Americans thought that West Germany and East Germany were both more similar to each other than Sri Lanka and Nepal and less similar to each other than Sri Lanka and Nepal. How is that possible?

25. First, when considering the question ‘which are more similar?’ the subjects looked for all the positive evidence that West Germany and East Germany were similar, and all the positive evidence that Sri Lanka and Nepal were similar. Since Americans know nothing about Asian countries, they had no positive reason to think Sri Lanka and Nepal were similar.

26. Similarly, when asked ‘which are more different?’ the subjects considered the positive evidence that West Germany and East Germany were different and the positive evidence that Sri Lanka and Nepal were different. Again, having no knowledge of Sri Lanka or Nepal, Americans chose (1), because of all the positive evidence in its favor.

27. But it cannot be true that East Germany and West Germany are both more similar and more different than Sri Lanka and Nepal. What the subjects did not do is consider the relevant negative evidencethat would disconfirm their hypotheses.

28. The Problem of Absent Data Sometimes it’s not just that we only look for or evaluate the positive evidence, but that there is no negative evidence. This can lead us to think we have very well-confirmed beliefs when we do not.

29. Hiring Job Applicants Suppose you’re hiring job applicants for your shoe company. You think people who haven’t studied a musical instrument would not be good employees. Who do you hire? The people who have studied music, of course! And if they’re successful at your company, do you have good reason to believe that you were right?

30. No! You have positive evidence– people you predicted would be successful, who are successful– but you have no negative evidence. What about all the people you didn’t hire, the ones who didn’t study music? They might have been successful too. They could’ve been more successful. You don’t know.

31. Absent evidence is all around us. Suppose you decide to major in accounting instead of philosophy. You find that you are very happy studying accounting. Did you make the right choice? You can’t know. You could have been more happy studying philosophy. There’s just no evidence.

32. Self-Fulfilling Prophecies

33. The Prisoner’s Dilemma There are two general strategies to playing the prisoner’s dilemma. You can view the game as one where the goal is for everyone to do well, and thus play “cooperatively” or you can view the game as one where the goal is for you to do better than your opponent, and thus play “competitively”.

34. The Prisoner’s Dilemma Believing that the goal is selfish, to win more for yourself, is a self-fulfilling prophecy. If you play against other selfish players, they will of course play competitively. But even if you play against cooperators, they will have to play competitively, or get 0 every round. So it will seem as if there are no cooperators.

35. Another Example Suppose you think I’m not a very nice person, so you avoid me. If you avoid me, then you’ll never have a chance to correct your initial impression. So if I’m a nice person, you’ll never find out if you start out thinking I’m not.