Randomization and Controls

1. Randomization and Controls

2. Causation and Correlation Last time we learned that when A and B are two correlated variables, there are four possible explanations: • A causes B • B causes A • Something else, C, causes both A and B • It’s just a coincidence/ accident

3. Observational studies reveal correlations between two variables, and these correlations are consistent with any of our four possible explanations. However, in the news, the correlations are often presented in causal terms, claims like “A causes B” even though that hasn’t been shown.

4. Eggs and Arteries According to the article “Egg Yolks almost as bad as smoking,” Dr. David Spence found that “people who eat egg yolks regularly have about 2/3 as much plaque buildup as smokers.”

5. A Causal Claim Spence is quoted in the article saying, “Just because you are 20 doesn’t mean egg yolks aren’t going to cause any trouble down the line.” He’s clearly suggesting that eggs cause plaque build up.

6. The theory outlined in the article is that eggs contain lots of cholesterol.

7. Alternative Explanations But are there other explanations? It’s very unlikely that plaque in your arteries causes you to eat more eggs. (B causes A) However, it’s possible that there is a common cause of egg-eating and plaque. (C causes A and B)

8. Eggs and Bacon Eggs are often served with unhealthy meals– omelets with bacon and cheese, cheeseburgers, fried rice, greasy Korean soup… Maybe it’s the things we eat with eggs that cause the plaque build up.

9. Chocolate and Nobel Prizes In the article, “Eat more chocolate, win more Nobels?” Dr. Franz Messerli claims to have found "a surprisingly powerful correlation” between the chocolate consumption in a country and the Nobel rate.

10. Chocolate and Flavanols The theory outlined in the article is that chocolate contains flavanols; flavanols slow down age-related mental decline (though this is doubtful); and…? Well, it’s not really explained how lessened mental decline makes you more likely to win Nobels. Wouldn’t chocolate have to make you smarter and not just prevent you from being dumber?

11. B causes A Dr. Messerli, according to the article, admits that “it’s possible… that chocolate isn't making people smart, but that smart people who are more likely to win Nobels are aware of chocolate's benefits and therefore more likely to consume it.”

12. C causes A and B The article also quotes Sven Lidin, the chairman of the Nobel chemistry prize committee: “I don't think there is any direct cause and effect. The first thing I'd want to know is how chocolate consumption correlates to gross domestic product.” He seems to be suggesting that GDP causes higher chocolate consumption and more Nobel prizes.

13. The GDP Theory Here’s what I think Lidin is suggesting: Chocolate is a luxury. Wealthy individuals are more likely to be able to afford it. Education is also a luxury. Poor people can’t afford to go to college for 10 years to get a PhD in chemistry. But you can’t win the Nobel prize in chemistry unless you’re a chemist. So he expects that the GDP or “wealth” of a country will be correlated both with chocolate eating and with Nobel prizes.

14. The GDP Theory So he expects that the GDP or “wealth” of a country will be correlated both with chocolate eating and with Nobel prizes. Wealth causes chocolate eating & Nobels.

15. Spurious Correlation It’s also possible that Dr. Messerli has committed the ecological fallacy, assuming that a correlation between a country’s chocolate consumption and that country’s number of Nobel prizes means that there is a correlation between individual chocolate consumption and individual Nobel-prize winning.

16. Ecological Fallacy Explanation Maybe smart people tend to avoid chocolate, because they know it can cause obesity. When they live in a country that consumes lots of chocolate they have to exercise their will power frequently. And maybe smart people + strong willpower = more Nobels. So it’s not eating chocolate but avoiding chocolate that causes Nobel prizes.

17. Cheating & Economic Dependence In “Men More Likely to Cheat If They Are Economically Dependent On Their Female Partners, Study Finds” a correlation is found between a man’s economic dependence on their spouse and his likelihood of cheating on her.

18. Cheating & Economic Dependence The study, authored by ChristinMunsch, a sociology Ph.D. candidate at Cornell University, found: “men who were completely dependent on their female partner's income were five times more likely to cheat than men who contributed an equal amount of money to the partnership.”

19. Controlling for Other Factors Munsch does not conclude that economic dependence directly causes cheating. He finds: “The relationship between economic dependence and infidelity disappeared when age, education level, income, religious attendance, and relationship satisfaction were taken into account.”

20. Root Causes That means that if two men have • The same age • The same education level • The same income • The same level of religious attendance and • The same relationship satisfaction Then they are equally likely to cheat, regardless of their economic dependence on their spouse.

21. A causes C, C causes B So the idea is that economic dependence affects one of these variables. For example: High economic dependence ↓ Low relationship satisfaction ↓ Higher rates of cheating

22. B causes A Are there alternative explanations? Well, it doesn’t make a lot of sense to suppose that cheating on your partner causes high economic dependence on your partner. After all, the study found that women who are highly economically dependent are less likely to cheat.

23. C causes A and B Still, there are other possible explanations of the data. It might be that old people are more traditional. Older men are less likely to be dependent on their female partners, because this is not traditionally acceptable. In addition, older men are less likely to cheat, because they have traditional values about marital fidelity.

24. The Age Theory Younger people are less traditional. Young men are more likely to be economically dependent on women and more likely to cheat: Economic Dependence ↑ Age ↓ Cheating

25. Quitting Smoking & Beating Cancer According to the article, “With lung cancer, quitters do better than smokers”: “Younger people with advanced lung cancer who quit smoking more than a year before their diagnosis survive longer than those who continue smoking, according to a new study.”

26. Quitting Smoking & Beating Cancer The study finds: “Among smokers with stage 1 or 2 lung cancer, for instance, 72% survived at least two years, compared to 93% of the never-smokers and 76% of people who'd kicked the habit a year or more before diagnosis. Only 15% of smokers with stage 4 disease survived two years, while 40% of never-smokers and 20% of former smokers did.”

27. Smoking Makes Cancer Worse? One interpretation is that smoking while you have cancer makes the cancer worse, or makes tumors grow faster. People who quit don’t have this effect. But the study also found: “After adjusting the numbers for factors such as age, race and radiation treatment, the researchers determined that quitters were just as likely to die from the early-stage cancers as were current smokers.”

28. Other Factors This means that two people, at the same stage of lung cancer, who also had: • The same age • The same race • The same amount of radiation treatment Were equally likely to die of cancer, even if one had quit smoking and the other had not.

29. A Morbid Possibility Here’s a possible common cause: a desire for death. People who wish they were dead are more likely to keep smoking and less likely to undergo radiation therapy when they find out they have cancer. People who want to live are more likely to quit, and more likely to undergo radiation treatment.

30. The Death-Wish Theory Smoking ↑ Desire for Death ↓ No Radiation Therapy

31. Overall Lessons Observational studies only reveal correlations, not causation. These studies are often reported in causal terms, which is misleading. There are often alternative explanations for the data, but we can’t know whether they are true either.

32. Controlling and randomization

33. Controlled Experiments Suppose I believe that eating chocolate makes you smarter. Maybe I have some evidence, in the form of observational studies that show a correlation between chocolate consumption in a country and the number of Nobel prizes won by that country.

34. But there are alternative theories: • Smartness causes chocolate eating • Wealth causes smartness and chocolate eating • Chocolate avoiding causes smartness • Etc.

35. Experimental Design I can rule out these possibilities with a well-designed experiment. What I want is two groups: one group (the experimental group) that eats chocolate because I tell them to, and another group (the control group) that does not eat chocolate, because I tell them not to.

36. Not: B causes A If the experimental group improves in intelligence over the course of the experiment, I know that this is not because higher intelligence leads to more chocolate consumption (even if that is true). In my experiment, intelligence does not cause chocolate consumption, I do. I am the experimenter and I say who eats chocolate.

37. Controlling for Additionally, if I make sure to put equal numbers of rich people in both groups, and equal numbers of middle-class people, and equal numbers of poor people, then I can make sure that improvements in the experimental group are not due to wealth: both groups have the same distribution of wealthy and non-wealthy people. This is called controlling for wealth.

38. Randomization Ideally, an experiment controls for as many variables as possible. To a large extent, this is done by randomly assigning individuals in the study to either the control group or the experimental group. This way, the members of the group are less likely to share features other than chocolate eating.

39. Randomization Randomization is not the only tool for controlling for confounding variables, and for certain variables, it can’t help. For example, suppose I want to test whether seeing pictures of babies makes people happier.

40. Babies and Happiness I randomly assign participants in the study to the control group and the experimental group. The control group takes a happiness questionnaire. The experimental group looks at baby pictures and then takes a happiness questionnaire.

41. A Crucial Difference Suppose that the experimental group rates higher on the happiness questionnaire. Does this mean that baby pictures cause happiness? No. The control group didn’t get to look at any pictures. Maybe they got bored, and boredom makes you less happy. We should give the controls pictures other than babies.

42. Maximal Similarity In general, the control condition and the experimental condition should be as similar as possible, and differ only in the variable being tested. For example, if the experimental group is given the happiness test by a beautiful woman and the control group is tested by a grumpy professor, that might be the real reason for a difference in scores, not the baby pictures.

43. It can be difficult or impossible to make the control and experimental conditions similar. If you want to study the effects of exercise, how do you make exercise and non-exercise similar?

44. Blinding and double-blinding

45. “Blinds” In experimental studies, we say that the participants are blind if they do not know which group they are in: the control group or the experimental group. Again, it’s not always possible to have blind participants, but this is considered best practice.

46. The Placebo Effect Why is blinding important? For several reasons: First, the mind has a pretty powerful effect on the body. People who think they’re receiving an effective treatment (even if they aren’t) are more likely to get better People who think they’re not receiving an effective treatment (even if they are) are less likely to get better.

47. Placebo Controls So we don’t test a drug by giving it to the experimental group and giving nothing to the control group. Then the control group knows it’s the control group, because it gets no pills! Instead, we give the control group fake pills with only sugar in them, and we don’t tell them that the pills are fake.

48. Subject Bias A second reason that blinding participants is good is that people who believe they are getting an effective treatment are more likely to say they’ve gotten better (even if they haven’t) and people who believe they are not getting an effective treatment are more likely to say that they haven’t gotten better (even if they have).

49. Subject Bias So, for example, if you give a group of people a fake pain medication and ask them whether it helps their pain, they might reason: “Well, I’m supposed to feel better, so I probably did get a little bit better.”

50. Deception and Blinding One common way of making sure subjects don’t know which condition they’re in is by lying to them about what you’re studying. You might tell people that you’re studying math ability, when what you’re really doing is studying the affects of cold rooms on math ability.