Cognitive Biases 4

1. Cognitive Biases 4 Fallacies Involving Probability

2. Conjunction Fallacy Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. Which is more probable? 1. Linda is a bank teller. 2. Linda is a bank teller and is active in the feminist movement.

3. Conjunction Fallacy The correct answer is (1): Linda is a bank teller. Suppose we take all the people in the world who fit the description (20,000, let’s say), and that amounts to 10,000 people. And suppose that among those only 1 is not active in the feminist movement. 9,999 bank tellers who fit the description are active in the feminist movement.

4. Conjunction Fallacy Then the probability that Linda is a bank teller is 10,000 out of 20,000 or 50%. And the probability that she is a bank teller AND active in the feminist movement is 9,999 out of 20,000, or slightly less than 50%.

5. This holds in general. If there are N people who fit the description, and M bank tellers who do, then N ≤ M and N/M is the percentage of bank tellers who fit the description. If X of the N bank tellers who fit the description are not feminists, then (N – X) is the number of feminist bank tellers who fit the description,

6. If X of the N bank tellers who fit the description are not feminists, then (N – X) is the number of feminist bank tellers who fit the description, and (N – X)/ M is the percentage of feminist bank tellers out of those who fit the description.

7. As a matter of mathematics, (N – M)/ X ≤ M/ X. There are always at least as many (and perhaps more) bank tellers who fit the description as feminist bank tellers who fit the description.

8. Always, the probability of two events happening (Linda being a bank teller AND Linda being a feminist) is less than the probability of just one of those events happening (for example, Linda being a bank teller). The illusion that the opposite is true especially occurs in cases where one event explains the other.

9. For example, suppose I tell you that there is a man named “George.” George turned water into wine, healed the sick, brought a dead person back to life, and came back to life himself after he died. What is the probability that George did all these things? How likely is it?

10. You can say whatever you like. 1%, 10%, 99%. But suppose I add to the story. I say “George was the son of God. That’s why he had all these powers.”

11. Many people will say that it’s more likely that George was the son of God AND did all these things than it is that he did all these things. But that can’t be true. “A & B” is always less (or equally) probable than A, or than B. For A & B to happen, A has to happen and also B has to happen.

12. Debiasing We can avoid this bias if we ask the question differently: There are 100 persons who fit the description above (that is, Linda’s). How many of them are: Bank tellers? ____ of 100 Bank tellers and active in the feminist movement? ____ of 100

13. This shows that it’s good to translate percentages and probabilities into frequencies (number of X out of number of Y). We are less susceptible to representativeness bias when things are phrased in this way.

14. Representativeness Our (false) judgment that Linda is more likely to be a feminist bank teller than to just be a bank teller is an example of how we judge the truth of claims based on how “representative” they are.

15. Consider again our case of coin flips that seem non-random, due to clustering. Since coins land 50% heads and 50% tails, “XO” and “OX” are representative of this even split, whereas “XX” and “OO” don’t represent it. So sequences with clustering seem non-random, even if they are (random).

16. Representativeness influences our other judgments as well. It’s hard to accept that two very tall parents tend to, on average, have less tall children (as regression to the mean requires). Children who are as tall as their parents are more representative of their parents’ heights.

17. Representativeness is often a good heuristic. A heuristic is a strategy that is easy to use in problem solving but doesn’t always work when applied. There is often no good reason to distinguish between heuristics and biases.

18. Representativeness is a good heuristic (sometimes) because (sometimes) things are representative.

19. Sometimes small effects have small causes. Burnt toast can be caused by leaving bread in the toaster for too long. • Sometimes complex effects have complex causes. World War I (a complex effect) was caused by a very complex set of factors, only one of which was the assassination of Archduke Ferdinand.

20. However: • Sometimes large effects have small causes. An outbreak of a disease may be caused by a tiny virus or bacterium. • Sometimes complex effects have simple causes. For instance, introducing a foreign species into a new land may cause radical changes in the ecosystem.

21. Base Rate Fallacy Suppose for a moment that ½ million people in Russia are affected by HIV/ AIDS, and that there are 150 million people in Russia. So the rate of HIV/ AIDS cases is 1 in 300. The government decides this is bad and that they should test everyone for HIV/ AIDS.

22. They develop a test with the following features: • If someone has HIV/ AIDS, then 95% of the time the test will be positive (correct), and only 5% of the time will it be negative (incorrect). • If someone does not have HIV/ AIDS, then 95% of the time the test will be negative (correct), and only 5% of the time will it be positive (incorrect).

23. Suppose you are a Russian who gets tested for HIV/ AIDS under the government program. The test comes out positive. How likely are you to have HIV/ AIDS? Most people will say something like 95%. After all, the tests are 95% correct, right?

24. This is not true. Remember that there are 150 million people in Russia, and they’re all getting tested. Only ½ million of them have HIV/ AIDS. So 149.5 million people do not have HIV/ AIDS. If you give the test to someone without HIV/ AIDS, it gives the correct result (negative) 95% of the time.

25. 149.5 million people x 99.5% true negative rate = 148.8 million people correctly diagnosed as not having HIV/ AIDS. Thus there are 149.5m – 148.8m = 700,000 people incorrectly diagnosed as having HIV/ AIDS who do not have it.

26. Furthermore, ½ million people actually do have HIV/ AIDS. If you give the test to someone who has HIV/ AIDS, it returns positive 95% of the time, and negative 5% of the time. So if all ½ million people are tested, 475,000 will be correctly diagnosed as positive, while 25,000 will be incorrectly diagnosed as negative.

28. So if you test positive for HIV/ AIDS, your chances of having HIV/ AIDS = • The number of people who have HIV/ AIDS and test positive ÷ the number of people who test positive = • True positive ÷ (true positive + false positive) = • 474,000 ÷ (475,000 + 700,000) = • 40%

29. Whether a test is good or worth doing depends not only on how accurate it is (95% true positive, 95% true negative), but also on how prevalent the condition being tested for is. Very rare conditions require very sensitive tests, whereas very prevalent conditions only need minorly accurate tests.

30. Consider an even more rare case: the number of terrorists who fly on planes. There are about 50 million passengers per year who fly through Hong Kong International Airport. There are about 10 airplane hijackings per year, but those are spread out over the 40,000 airports in the world.

31. Let’s assume there’s all 10 terrorists in the entire world fly through HKIA. Now suppose the government introduces a “99% accurate” terrorist detection test:

32. If someone is a terrorist, 100% of the time the test is positive = terrorist. • The other 0% of the time (never) the test is negative = not a terrorist. • If someone is not a terrorist, 99% of the time the test is negative = not a terrorist. • The other 1% of the time the test is positive = terrorist.

33. Now imagine that the government applies the test to everyone who flies through HKIA for an entire year. There are 50 million minus 10 terrorists who fly every year. So the test will correctly conclude that (49,999,990 x 99%) = 49,499,990 ∼ 49.5 million of them are not terrorists.

34. But it will incorrectly conclude that of those 49,999,990 people who are not terrorists, (49,999,990 x 1%) = 499,999 ∼ ½ million of them are terrorists! Almost half a million innocent people classed as terrorists by the “99% accurate” detection device!

35. And what about the terrorists? The test will identify them correctly 100% of the time: 10 x 1.00 = 10 correct identifications and identify them incorrectly 0% of the time: 10 x 0 = 0 incorrect identifications. Doesn’t that mean it’s a good test? Won’t we always catch the terrorist?

36. No! Suppose someone, Mr. X tests positive. How likely is it that Mr. X is a terrorist? There are 499,999 “false positives” (non terrorists that the test says are terrorists) and 10 “true positives”. So if you test positive, your chance of being correctly identified as a terrorist is:

37. So if you test positive, your chance of being correctly identified as a terrorist is: • True positive ÷ (true positive + false positive) = • 10 ÷ (10 + 499,999) = • 10 ÷ 500,009 • 0.00001999 (about 2 in 100,000)

38. Even with a test that is 100% accurate at catching terrorists and 99% accurate at letting non-terrorists go free, we are still left with more than half a million “positives” (= people who have tested positive for terrorism), only 10 of whom are actually terrorists! We need a new test to sort these people!

39. Base Rates The “base rate” is the percentage of people in the population who have a certain property. The base rate of terrorists is the percentage of terrorists in the population, the base rate of HIV/AIDS cases is the percentage of people who have HIV/AIDS in the population, etc.

40. Base Rate Neglect The “base rate neglect fallacy” is the fallacy of ignoring the base rate when making a judgment. For example, if I assumed you were a terrorist, because you tested positive, I would be committing the base rate neglect fallacy. I should assume you’re still probably not a terrorist.

41. Base Rates As we have seen, base rates matter. If the base rate of a condition is very low (small percentage of terrorists), then even very accurate tests (100% true positive, 99% true negative) can be useless. In our example only 2 in 100,000 people who tested “positive” for terrorism were terrorists.

42. Base Rate Neglect The base rate neglect fallacy has nothing at all to do with the number of people there are in the world. Nothing. It has to do with the probability of a variable taking on a certain value, for instance, the probability that someone’s height = 1.5m, the probability that terrorist = true (someone is a terrorist)…

43. Base Rates This is the “base rate” of people who are 1.5m tall, and the “base rate” of terrorists. If 1 in 100 people are terrorists, then the rate of terrorists is 1 in 100 and the probability that a randomly selected person is a terrorist is 1 in 100.

44. Base Rates We call this the base rate, because it is the probability that someone is a terrorist when we don’t know anything else about them. It might be that the base rate of terrorists is 1 in 100, but the rate of terrorists among people who are holding rocket launchers is 1 in 2, and the rate of terrorists among retirees is 1 in 500.

45. Tests The base rate neglect fallacy happens when we have a test that is meant to detect the value of a variable. For example we might have a test that tells us whether someone has AIDS or not, or whether someone is driving over the speed limit, or whether they are drunk.

46. Reliability of Tests Here is the important, and crucial fact. Please learn this: As the base rate of X = x decreases, the # of false positives on tests for X = x increases. Tests are less reliable when the condition we are testing for becomes rare (low base rate).

47. Base Rate Neglect Fallacy The base rate neglect fallacy happens when: • There is a low base rate of some condition. • We have a test for that condition. • Someone tests positive. • We assume that means they have the condition, ignoring the unreliability of tests for conditions with low base rates.

48. Base Rate Neglect Humans have a tendency to ignore base rates. For example, Kahneman and Tversky (1973) conducted a study in which participants were supposed to estimate the GPAs of certain (fictional) students.

49. Kahneman & Tverskey 1973 Some of the participants were given good evidence that students had high (or low) GPAs. In particular, they were given the students’ percentiles (95th percentile, for example). Other participants were given only very weak evidence: the scores that the students got on a test of humor.

50. All the participants were given the base rate of students with various GPAs. For example, 20% A, 40% B, 30% C, 10% D. But all of the participants ignored the base rate.