1 / 36

Statistics as a Distillation of Everyday Experience

Statistics as a Distillation of Everyday Experience. Gerald van Belle Department Biostatistics, Department of Occupational and Environmental Health Sciences University of Washington, Seattle, WA. Where Are We Going?. Statistics as distillation of everyday experience

benoit
Download Presentation

Statistics as a Distillation of Everyday Experience

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistics as a Distillation of Everyday Experience Gerald van Belle Department Biostatistics, Department of Occupational and Environmental Health Sciences University of Washington, Seattle, WA. KNAW Lecture

  2. Where Are We Going? • Statistics as distillation of everyday experience • 1.Variation 2. Causation • Experience can benefit from everyday statistics KNAW Lecture

  3. KNAW Lecture

  4. A. Variation in everyday experience • Describing and classifying variation • Selection in the face of variation • Controlling variation • Inducing variation • “Missing data” KNAW Lecture

  5. Statistician drowning in river of average depth 25 cm KNAW Lecture

  6. 1. Describing and classifying variation • We tell stories of ab”normality”* Air travel horror stories, laptop disasters,… • We sort into genres: art, biology, literature* Concept of “population”* Characteristics of population and “sample” • Variation in time, space, social structures,…* Waves on beach (non-stationarity)* Hierarchy, “social class” • We make inferences based on limited data* And often get the wrong population* Basis for a great deal of humor* Switch in “expectation” KNAW Lecture

  7. 2. Selection in the face of variation • Need to know selection mechanismSpend very little time on this,Assumption of “missing at random” • Error of thinking that current observation is representative • Unintended intentional selection • Two examples: KNAW Lecture

  8. KNAW Lecture

  9. KNAW Lecture

  10. 2. Selection in the face of variation--2 • Need to know selection mechanismRandom selection as gold standard • “Representativeness”* Kruskal and Mosteller papers* Slippery concept* Large sample vs small sample KNAW Lecture

  11. 3. Controlling variation • Clearest examples in sports: Divisions, “junior”,… • Societal examplesMin, max speed limitsOccupational (noise limits, flying hours)Vergunningen, vergunningen,… • “Blocking” in statistics KNAW Lecture

  12. 4. Inducing variation • Antitrust laws *Increase competition, i.e. variability • Draft system in sports* Teams more equal, P(win) near 1/2 • Societal* Admission to medical school in Holland* Representativeness (again)* Key to clinical trials KNAW Lecture

  13. 5. “Missing” data • Serious problem, obviously • Anatomy of missingness • Normal (e.g. pediatrician chart) • Transcription error • Just not there (Murphy was here) • Deliberately missing (e.g. extended testing on subset of patients) • Impacts population of inference • Example: KNAW Lecture

  14. KNAW Lecture

  15. Vulnerability Analysis of Spitfires (sample: 15/400) KNAW Lecture

  16. * * * * * * * * * * * * * * Composite of hits Abraham Wald Advice: KNAW Lecture

  17. Another anatomy of missingness • …as we know, • there are known knowns; • there are things we know we know. • We also know there are known unknowns; • that is to say, • we know there are some things • we do not know. • But there are also unknown unknowns— • the ones we don’t know we don’t know. • Donald Rumsfeld (set to music, see NPR website) KNAW Lecture

  18. Translation into modern statistics Non-missing MCAR/MAR Non-ignorable • …as we know, • there are known knowns; • there are things we know we know. • We also know there are known unknowns; • that is to say, • we know there are some things • we do not know. • But there are also unknown unknowns— • the ones we don’t know we don’t know. • Donald Rumsfeld KNAW Lecture

  19. B: Causation in everyday experience • Aristotle’s four causes • Hardwired to look for causation • Hardwired to assume association is causation • Hardwired to assign blame (secondary causes) • (The Dutch: “oorzaak” is closer to Aristotle’s aition) KNAW Lecture

  20. 1. Aristotle’s four causes • Material cause(table made of wood) • Formal cause (four legs and flat top make this a table) • Efficient cause(carpenter makes a table) • Final cause (surface for eating or writing makes this a table)(From S.M. Cohen, U Washington) KNAW Lecture

  21. Four questions • What is the question? was • Is it testable? Was • Where will you get the data? did • What will the data tell you? do The great divide KNAW Lecture

  22. 1. What is the Question? Is it testable? Not everything that can be counted counts and not everything that counts can be counted. Albert Einstein KNAW Lecture

  23. KNAW Lecture

  24. Testable hypothesis but What was the question? 2. Frequent Consulting Scenario KNAW Lecture

  25. Example from Science (February 23, 2007) • Title Redefining the age of Clovis • Front page Flints (pictures) • Page 1045 Summary paragraph • Page 1067 News story • Page 1122—1126 Article (numbers) • Very different “flavor” for each section KNAW Lecture

  26. Question Testable version KNAW Lecture

  27. 3. Hardwired to look for causation • Story • Instinctive looking for causes • Challenging in courts crime in search of criminal, stock market (“if only I had bought Microsoft in 1980”), and science (global warming) • Life forces us to do this ex post factoWhitehead quote KNAW Lecture

  28. 4. Hardwired to assume association is causation • Story • Criteria for causation help • Causation in observational studies is a great challenge • R.A. Fisher introduced randomizationas way of establishing cause-effect KNAW Lecture

  29. 5. Hardwired to look for secondary causes • Assists in soccer, hockey, basketball • Tendency to blame (1) immediate efficient cause to more distal(Adam→Eve→snake maneuver )(2) move from efficient cause to material or formal cause; that is, change the “universe of discourse” • Surrogate outcomes in science (great work by Ross Prentice) KNAW Lecture

  30. 6. Challenges to Causation in Observational Studies • Selection bias “Where did you get the data?” • Confounding • “What do you think the data are telling you?” • Interplay of selection bias and confounding • Effect modification needs to be considered KNAW Lecture

  31. 7. Observational vs experimental studies • Characteristic Observational Experiment • Ethical issues Fewer More • Orientation Retrospective Prospective • Inference Weaker Stronger • Selection bias Big problem Less • Confounding Present Absent • Realism More Less • Causal plausibility Weaker Stronger • Researcher control Less More • Analysis More complicated Less KNAW Lecture

  32. Consequence • In view of • Variation • Undocumented selection • Hard-wired tendencies for causation • Observational data • We have a lethal mixture for dealing with important scientific questions affecting daily life: KNAW Lecture

  33. APHA Journal, May, 2004 KNAW Lecture

  34. So we always need to ask: • What is (was) the question? • Is (Was) it testable? • Where will (did) you get the data? • What do you think the data are telling you? KNAW Lecture

  35. Variation Causation Experience can benefit from everyday statistics • Variation is fact of life • “Population” as model • Representativeness • Regression to the mean • What is the question? • Testable question? • Association or causation • Causation through randomization KNAW Lecture

  36. Hartelijkbedankt voor Uw aandacht KNAW Lecture

More Related