Statistics as Evidence

1. Statistics as Evidence Arguing by the numbers

2. impressing with statistics • approximately one in five children ages 10-17 who use the Internet frequently have received a sexual solicitation or approach over the Internet. • the mortality rate for avian flu is more than 50 percent. • The average L.A. driver spends 136 hours per year in rush hour traffic • A typical child has seen 4,286 acts of violence on TV by age 18. • Bill Gates makes more money than the poorest 100 million Americans

3. But are the numbers really that impressive? • An article in the New England Journal of Medicine stated “a person who purchases a handgun is 57 times more likely to commit suicide within a week of buying the weapon than the general population as a whole.” • The study said suicide is the leading cause of death among gun buyers, in the first year after a weapon is purchased. • The statistic is misleading, however, because buying a handgun doesn’t make one more likely to commit suicide, wanting to commit suicide makes one more likely to buy a handgun. • People who are bent on committing suicide probably buy more knives, rope, sleeping pills, and razor blades, too.

4. Kevin & Bean, KROQ 10/9/08: “Tigers have killed kill 300,000 people in India.” Wikipedia: “Although humans are not regular prey for it, the tiger has killed more people than any other cat. Between 1800 and 1900, it is estimated that tigers had killed over 300,000 people in India alone” Actually, about 300,000 people live in the Sundarban area of Bangladesh, where Royal Bengal tigers are found Tigers are responsible for about 10 deaths a year. (Reuters, Jan 23, 2008, reported by Enamul Haque) Valmik Thapar, a leading expert on tigers in India, put deaths due to tiger attacks at about 80 people annually. (The Statesman, New Delhi, July 16, 2001) Are the numbers accurate?

5. On average men think about sex every 7 seconds. 54% of men think about sex everyday or several times a day, 43% a few times per month or a few times per week, and 4% less than once a month (Kinsey Report from KinseyInstitute.org) Are the numbers accurate?

6. watch out for “unknowable” statistics • example: The average person tells 13 lies per week • example: “in up to 98% of the cases, simply brandishing a gun is sufficient to stop a crime.” John Lott Jr., Wall street Journal July,30, 2001 • example: Judy Garland sang “over the Rainbow” 1,476 times.” Esquire, Feb. 1964

7. comparing non-comparable units (apples versus oranges) A study conducted by the dean of admission for Harvard University concluded that Harvard students who took prep courses scored lower than students who didn’t take prep courses. The study concluded “the coaching industry is playing on parental anxiety.” • However, the comparison is not based non-equivalent samples of students. Students who didn’t even bother taking a prep course might be much stronger academically than students who opted to take such a course.

8. watch out for unrepresentative samples • Non-random samples: • voting on the American Idol show • phone calls or emails to a politician about a particular bill • testimonials on infomercials • talk radio callers • To be truly valid a sample must be: • random • representative • sufficient

9. questionable extrapolations • Using a limited sample (limited in time or number) and extrapolating the percentages nationwide or globally • example: According to the National Self Defense Survey in 1994, the rate of Defensive Gun Uses can be projected nationwide to approximately 2.5 million per year -- one Defensive Gun Use every 13 seconds. • http://www.pulpless.com/gunclock/framedex.html

10. keep the numbers in perspective • Compare “scary” statistics to other, well-established health risks. • “mad cow disease” or the “flesh eating virus” sound scary, but the odds are much greater of dying of dog bites, drowning in the tub, or being struck by lightning. • Chicken pox kills 100 people per year in America. • Flu kills about 35,000 people annually in the U.S.

11. School killings Total school-related violent deaths, August 1, 2005, July 31, 2006: Shooting: 15Suicides: 1  Murder-Suicide: 4 Fight-Related: 0 Stabbing: 3 Other: 4 Total = 27 Compared to drownings and car crashes In 2003, 782 children ages 0 to 14 years died from drowning (CDC 2005). In the United States during 2004, 1,638 children ages 14 years and younger died as occupants in motor vehicle crashes Keep the numbers in perspective

12. misleading statistics • Some studies have reported that 90% of students are bullied at school. • But the results depend on what the researcher defines as “bullying.” Does bullying included physical and verbal abuse? If bullying means “people being mean to you,” then it is surprising that 10% of students haven’t been bullied!

13. misleading graphs and charts • This CDC graph, comparing causes of death is misleading, because it leaves out deaths from heart disease, cancer, and stroke. The omission tends to make smoking look much more harmful by comparison.

14. bias in gathering and reporting statistics • Cathie Adams, president of the Texas Eagle Forum, stated on CNN on April 21, 2005 that: “research shows that children in same-sex couple homes are 11 times more likely to be abused sexually…It is a proven fact and that was a research study done in the state of Illinois” • But Illinois doesn’t keep track of whether households are same sex or not. It's possible that much of what the study calls “homosexual abuse” occurred in heterosexual households. • The study also confuses pedophilia with homosexuality. If the sexual abuse was male-male or female-female, the study presumed the abuser was gay or lesbian.

15. not defining the units to be counted clearly • example: statistics on “juvenile violence” • the legal ages of juveniles vary from state to state, as do the ages at which juveniles may be tried as adults. • example: statistics on “child abductions” • can refer to kidnappings by strangers or parental custody disputes

16. carefully considering statistical measures • Failure to report the margin of error (a.k.a “sampling error”) • example: +/- 5% in a national poll of 1,200 respondents • Making conclusions within the margin of error • If candidate A is preferred by 43% of voters, and candidate B is preferred by 46% of voters, all one can say is that they are tied in a statistical dead heat. • Not stating whether the results are statistically significant or what the threshold for statistical significance is. • example: p < .05

17. media slanting: What to watch and listen for • Remember: news media are after ratings, not necessarily the truth. • Watch for bias during the opening or lead-in. • Ask how representative or generalizable case studies are. • Don’t let pictures overwhelm the facts. • Beware of quoted sources with an agenda. • Beware of “black hats” or “white hats.” • When statistics are presented is the underlying methodology explained? • Watch out for rhetorical questions with no follow-up proof.