Interesting Statistical Phenomenon

2. Interesting Statistical Phenomenon VA San DiegoAddictions Seminar4/16/08Kevin Cummins

3. Definitions • Paradox: a statement that is seemingly contradictory or opposed to common sense and yet is perhaps true • Fallacy: a false or mistaken idea • Principle: a comprehensive and fundamental law, doctrine, or assumption

4. Outline • Objective • Simpson’s Paradox • Will Roger’s Paradox • Lord’s Paradox • Berkson’s Paradox • Monte Hall Paradox • Others

5. Objective • Create awareness of several statistical issues that might arise during your research

6. Outline • Objective • Simpson’s Paradox • Will Roger’s Paradox • Lord’s Paradox • Berkson’s Paradox • Monte Hall Paradox • Others

7. Simpson’s Paradox Occurs when the relationship between two categorical variables is reversed after a third variables is introduced. The relationship between two variables differs within subgroups compared to that observed for the aggregated data.

8. Which Airline Should You Fly?

10. America West Alaska Airlines .29 .17 .14 .11 .08 .05

11. Simpson’s Paradox: Remedies/Responses Study DesignUse ExperimentsCollect appropriate covariate data Know the Research System Collect appropriate covariate data Analytically introduce conditionals(i.e. moderators/covariates) Use appropriate interpretations

12. Outline • Objective • Simpson’s Fallacy • Will Roger’s Paradox • Lord’s Paradox • Berkson’s Paradox • Monte Hall Paradox • Others

13. Will Roger’s Paradox “When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states.”

14. The Will Rogers Paradox (WRP) is obtained when moving an element from one set to another set the average values of both sets increase: The effect will occur when both of these conditions are met: 1. The element being moved is below average for its current set. 2. The element being moved is above the current average of the set it is entering.

15. WRP: Effect Shifting One Observation on the Mean

16. WRP: Health Insurance Example PPO No Longer Free The 1997 migration moved lower cost PPO subjects into the HMO Young et al. 1999

17. Will Rogers: Remedies/Responses Know Your System In This Case: Statistically Adjust for Baseline Costs

18. Outline • Objective • Simpson’s Fallacy • Will Roger’s Paradox • Lord’s Paradox • Berkson’s Paradox • Monte Hall Paradox • Others

19. Lord’s Paradox • Situation where change score analysis and ANCOVA yield apparently conflicting results

20. A Simplified Example • Assessment of a supplemental educational program (tutoring) • 10 students from different schools • 5 schools opted into the programs (free-choice) • Pre and post assessments given • No random/sampling/measurement error (simplified)

21. Statistician One Calculates difference scores for each group Change scores the same for both groups Statistician Two Adjust for initial score Finds group differences Two Statisticians

23. Paired t-Test Statistician One Data: group 1 vs. group 2 t = -0.002, df = 299, p-value = 0.99 ANCOVA Statistician Two Coefficients: Value Pr(>|t|) (Intercept) 15.0 0.00 Pre 0.5 0.00 Group 20.0 0.00 Two Statisticians

24. Statistician’s Assumptions • Statistician one assumes that in the absence of any differential treatment effect the two groups despite different baselines would show equivalent changes • Statistician two assumes that in the absence of any differential treatment effect the change of the groups as a whole is the same as the change within groups • Both of these causal assumptions are untestable

26. Establish causal/system assumptions Use the best descriptive statement Use and report multiple approaches (Wright 2006) Know that ANCOVA has greater power Graph your data Lord’s Paradox: Remedies/Responses

27. Outline • Objective • Simpson’s Fallacy • Will Roger’s Paradox • Lord’s Principle • Berkson’s Paradox • Monte Hall Paradox • Others

28. Berkson’s Paradox An association reported from a hospital-based case-control study can be distorted If cases and controls experience differential hospital admission rates with respect to the suspected causal factor

29. Typical Berkson Example Example from Roberts et al. 1978 Investigated the relationship between circulatory and respiratory disease. Sampled the general population and hospital populations.

30. Circulatory Disease OR = 3.9 [95% CI: 1.4-10.9]

31. Circulatory Disease OR = 1.3 [95% CI: 0.9-2.3]

32. Real Berkson Example Example from Berkson 1946 Hypothetical example exploring the relationship between diabetes and cholecystitis No greater admission rate for subjects with multiple conditions Different rates of admission for cases and controls

34. Different Conditional Rates Not Required

35. Berkson’s: Remedies/Responses • There is no analytical mitigation • Limit conclusions • Consider alternative study design

36. Outline • Objective • Simpson’s Fallacy • Will Roger’s Paradox • Lord’s Principle • Berkson’s Paradox • Monte Hall Paradox • Others

37. Monty Hall Paradox

39. Lindley’s Paradox • Standard Sampling Theory VS. Bayesian Theory Under some circumstances strong evidence against the null hypothesis doesn’t result in the null being rejected

40. Benford’s Law Ones are the most common leading digit in most data. Notice that if a data entry (base 10) begins with a 1, the entry has to be at most doubled to have a first significant digit of 2. However, if a data entry begins with a 9, it only has to be increased by, at most, 11% to change the first significant digit into a 1.

41. Review Big PictureUse care to interpret observational studiesKnow your system Conditional ResponsesSimpson’sLord’sWill Roger’s Perspective ProblemsBerkson’sMonte Hall

42. Doctor Tyrano, Look for a Covariate!