How to Lie with Statistics

1. Election 2004 How to Lie with Statistics Chad Orzel Physics and Astronomy 10/5/04

2. Statistics are commonly used to deceive Technically true, but deceptive Preys on fear of numbers “Math is hard!” --Barbie False impression of accuracy “Figures never lie, but liars figure.” “There are three kinds of lies: Lies, Damned Lies, and Statistics.” --attributed to Benjamin Disraeli What’s This All About? • Need to know how to lie with statistics, to keep from being lied to with statistics.

3. 0) Fabrication Just make things up… Can be very effective: Lyndon Johnson: “Make the son of a bitch deny it.” Not what we’re talking about today Talking about ways to say things that are true, but misleading… Ways to Lie to Voters Swift Boat Veterans for “Truth”

4. Example: Election 2004 A typical person in this class: 1) Is Male 2) Plans to Vote for Kerry 3) Has two siblings 4) Is 26 years old 5) Made $18,000 last year All true statements, based on survey results!

5. 1) Omission  Leave Things Out Previous slide: What does “typical” mean? Specify what kind of average you’re using: Ways to Lie to Voters Mean: Add ‘em up, divide by total number Median: value in middle (half higher, half lower) Not the same

6. Nearly identical for random variables “Normal Distribution” “Bell Curve” Mean affected by extreme values Diverse populations Median less sensitive to extremes  Usually better for economic data Mean and Median Mean: 190.1 Median: 190 Very different for skewed data:

7. Few people with huge families Median  Pull mean up Mean Limited range  Can’t have < 0 siblings Example 1: Siblings Most people have 0,1,2

8. Students, mostly 19-22 (Much) older faculty Median Nobody at mean age Mean Very bad description Example 2: Age Diverse Population Problem

9. Median Usually where this lie comes up: “The average family will save $2,000 under my tax plan…” What kind of average? Mean Example 3: Income Sort of silly, really… Remember: The mean includes Bill Gates…

10. Kerry’s $9,000 Bush Tax Cut “We're told that jobs that pay $9,000 less than the jobs that have been lost is the best that we can do.” “111 million taxpayers will save, on average, $1,586 off their taxes.” Facts: Fact: 1) 25% receive NO cut Based on comparison of broad categories (drops mean to $1,217) Lost: Manufacturing jobs 2) Median cut: $470 Gained: “Service” jobs Half of all taxpayers get $470 or less (http://www.factcheck.org/article.aspx?docID=228) (http://www.factcheck.org/article.aspx?docID=145) Campaign Examples  Includes burger flippers

11. 1) Omission (Continued) The Fifth Dentist Problem “Four out of five dentists surveyed…” How many dentists total? 5 total: not a good sample Leave out the sample size, and you can prove just about anything… “Four out of five cards drawn from this deck were black!” Ways to Lie to Voters

12. “And that's what people are seeing now is happening in Afghanistan. Ten million citizens have registered to vote. It's a phenomenal statistic. That if given a chance to be free they will show up at the polls. Forty-one percent of those 10 million are women.” --G.W. Bush, 1st Presidential Debate • Ratio of men registered to women registered: 58.6 to 41.4 percent • Estimated eligible voting population in Afghanistan: 9.8 million • Registered voters in Afghanistan, as of August 21: 10.3 million • Reported number of registration cards a single Afghan has been able to obtain: from 2 to 40 • Percent of the estimated eligible male population that is now registered to vote: 120 percent • Number of provinces that are over-registered: 13 (out of 30) • Number of provinces which registered voters exceed the population by 40% or more: 4 (http://www.tcf.org/afghanistanwatch/main.htm#voterregistrationfraud) Campaign Example

13. Fear of big numbers: “My opponent wants to spend $2 million on [something]…” Sounds bad… $2 million = 1/1,000,000th of the budget = chump change Need to put big numbers in context Ways to Lie to Voters 2) Exaggeration Make Something of Nothing

14. Nothing false in graph Creates false impression Example: Guys Rule! More Survey Data… Scale axes to blow up small differences

15. Example: Guys Rule! Honest presentation: Full scale shown Bars same width, color Slightly more male students Not that big a difference

16. Example (http://www.pollkatz.homestead.com/)

17. What does margin of error really mean? (http://www.washingtonmonthly.com/archives/individual/2004_08/004536.php) Campaign Example “According to the first post-debate poll, from Newsweek, John Kerry leads President Bush by a margin of 49% to 46%. Put Nader in the mix and Kerry's margin drops from 3 to 2.” --Josh Marshall, Talking Points Memo (weblog) “In the first national telephone poll using a fresh sample, NEWSWEEK found the race now statistically tied among all registered voters, 47 percent of whom say they would vote for Kerry and 45 percent for George W. Bush in a three-way race.” --MSNBC (1,013 voters surveyed, Margin of Error +/- 4%)

18. 3) MisdirectionTrue, but Irrelevant Quote impressive statistics about side issues Creates false impression of real support 4) False Correlation Post Hoc Fallacy Homicide rates peak in summer Ice cream sales peak in summer Other Ways to Lie Therefore, ice cream leads to murder? Correlation is not Causation

19. 1) Who created it? Do they have an agenda? 2) Why was it created? Research or politics? 3) How was it created? Methodology What to Do? Questions to ask about any statistic:

20. 4) What’s missing? Is there hidden context? 5) Is it relevant? Avoid misdirection 6) Does it make sense? If it sounds ridiculous, it probably is… What to Do? (continued) Questions to ask about any statistic: