1 / 16

How to Lie With Statistics

How to Lie With Statistics. I'm so confident!. The universe is not one of clockwork Everything we measure has uncertainty How can we confidently draw conclusions and make decisions without sacrificing our intelligence?. Quantify Your Uncertainty!. Family of intervals to handle this

oona
Download Presentation

How to Lie With Statistics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. How to Lie With Statistics

  2. I'm so confident! • The universe is not one of clockwork • Everything we measure has uncertainty • How can we confidently draw conclusions and make decisions without sacrificing our intelligence?

  3. Quantify Your Uncertainty! • Family of intervals to handle this • Confidence • Prediction • Comprised of • Point estimator • Degree of Confidence • Uncertainty • Rely on • Underlying distribution (assumption) • Data collection

  4. Formula Z*,t* Margin of Error

  5. What could go wrong? • Poor data collection • "I talked to my 10 closest friends – Romney has got the election in the BAG!" • Wrong assumptions • Excessive uncertainty • Unrealistic expectations • Degree of confidence is too high/low • Wrong type of interval • Prediction versus Confidence

  6. Degree of Confidence • What are we confident in? • The Process • If we collect data from the same population, over and over and over again, we can expect our interval to contain the true (correct) value x% of thetime where x is ourdegree of confidence(confidence level) • "The feeling or belief thatone can rely on someoneor something"-- What I got Googling"What is Confidence"

  7. Degree of Confidence • What is confidence NOT? • A probability of any single interval containing the true value • The % of all population values contained in the interval • A product of your self-esteem

  8. Lancet Survey • October 29, 20041The Lancet published a study on the effects of the US invasion of Iraq, specifically on the number of additional civilian deaths due to the invasion • The data estimated 98,000 excess deaths • They were 95% confident that the true number of excess deaths was between 8,000 and 194,000 1Les, R., Lafta, R., Garfield, R., Khudhairi, J., & Burnham, G. (2004). "Mortality before and after the 2003 invasion of Iraq: Cluster sample survey. The Lancet, 1857-1864.

  9. Lancet Survey • October 11, 20062The Lancet published a follow-up study • The data now estimated 654,965 excess deaths • They were 95% confident that the true number of excess deaths was between 392,979 and 942,636 • Claimed that they asked for death certificates 87% of the time and that 92% of the time they asked they were shown a certificate (80% of reported deaths had a certificate to verify the death) • Iraqi Ministry of Health only issued ~50,000 death certificates for post-war violent deaths 2Burnham, G., Lafta, R., Doocy, S., & Roberts, L. (2006). Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey. The Lancet, Retrieved from http://www.brussellstribunal.org/pdf/lancet111006.pdf

  10. Statistical Ties • Des Moines Register Iowa Poll Results Published 9/29/2012 • Obama 49% • Romney 45% • Other 4% • Undecided 2% • Margin of Error ±3.8% • Who is winning? • Register claims Obama has a 4% lead • Statistically inconclusive • The poll results give an ~84% chance that Obama is really ahead

  11. Multiplicity of Error

  12. Multiplicity of Error • Dr. Elisabeth Targ made headlines when she identified an effect of remote faith healing • The study results were published in the peer-reviewed journal The Western Journal of Medicine under the title "A randomized double-blind study of the effect of distant healing in a population with advanced AIDS. Report of a small scale study" • Results from the double blind study showed that members of the control group spent 600% more days in the hospital and contracted 300% as many AIDS-related illnesses • Considered the most rigorous, scientifically valid experiment to demonstrate the effect of remote faith healing

  13. Multiplicity of Error • What was learned until after the fact: • Study was designed to look for differences in mortality rates • Triple-drug Anti-Retroviral therapy became common practice one month into the six month study • Only one person died – insufficient data to make conclusions • Adjusted the scope to look at HIV physical symptoms and Quality of Life • Also inconclusive • Adjusted scope again to look at Mood State scores • Control group performed better than treatment group • Adjusted scope again to look at Hospital Stays and Doctor Visits • Treatment group performed statistically significantly better • Adjusted scope to include at 23 illnesses associated with AIDS • Data hadn't been collected as part of the study design, so blindness was broken and data was aggregated from hospital charts • Sharpshooter fallacy • Spray bullets down range, then draw your target around a cluster

  14. Multiplicity of Error • Keeping ourselves honest • Bonferroni correction • Don't use α, use where m is the number of comparisons • Conservative • Holm-Bonferroni correction • Compare the smallest p-value to • If it is significant, continue testing • Otherwise, stop • Compare the next smallest p-value to • Continue comparing the p-values until one is not significant, then stop • Less conservative • Others that are implemented in statistics packages (Minitab, JMP, etc…) • Tukey's HSD • Dunnet's • And many more (you can't get a Ph.D. in statistics without inventing a correction to name after yourself)

  15. Biggest Lie Ever Told • Vitamin C prevents or helps cure the Common Cold • Linus Pauling (an incredibly bright, talented scientist) championed the need for megadoses of Vitamin C to prevent or cure the Common Cold (and cancer) with the publishing of his book Vitamin C and the Common Cold in 1970 • Had a whole institute setup to further study the benefits of megadosing Vitamin C • At least 30 experiments have been performed to test the ability of Vitamin C to prevent or cure the Common Cold in large groups of people (dates of experiments go from 1967-2001) • Not a single one has found a link between Vitamin C and the Common Cold

More Related