A Semi-Theoretician's Mid-Day Confession : The True Meaning of i.i.d. in (Applied) Statistics

1. A Semi-Theoretician's Mid-Day Confession:The True Meaning of i.i.d. in (Applied) Statistics Xiao-Li Meng Department of Statistics Harvard University

2. Crying and/or Smiling? • Size & Complexity of data are increasing; • Depth & Specificity of investigation goals are increasing; • Available Time for conducting analysis is decreasing. • Grand Challenges = Great Opportunities

3. My Two Tales of NLAAS • Statistics and Lies: How to use Bayesian modeling and multiple imputation to reduce response bias. • Disparities in Defining Disparity: Statistical conceptual frameworks (and theimpossibility of estimating disparity without making strong causal assumptions).

4. Overview of NLAAS National Latino and Asian American Study (NLAAS) • NLAAS, conducted in 2002-2003, and made public on July 2007, is a national psychiatric epidemiologic study conducted to measure psychiatric disorders and mental health service usage in a nationally representative household sample of Asians and Latinos. • There are more than 5,000 variables (and the number is still growing!) • Total sample size is 4864: 2554 Latinos + 2095 Asians + 215 Whites

5. A HUGE Questionnaire!

6. Statistics Can Lie But Can Also Correct For Lies: Reducing Response Bias Via Bayesian Imputation Joint work with Jingchen Liu1, Chih-nan Chen2, and Margarita Alegria2,3 1 Harvard University 2 Cambridge Health Alliance 3Harvard Medical School

7. *SR2. Have you ever in your lifetime been admitted for an overnight stay in a hospital or other facility to receive help for problems with your emotions, nerves, mental health, or your use of alcohol or drugs? • YES 1 • NO 5 GO TO *SR9.01 • DON’T KNOW 8 GO TO *SR9.01 • REFUSED 9 GO TO *SR9.01 • *SR5a. Was this in the past month, past six months, past year, or more than a year ago? • PAST MONTH 1 GO TO *SR5c • PAST SIX MONTHS 2 GO TO *SR5c • PAST YEAR 3 GO TO *SR5c • MORE THAN A YEAR AGO 4 • DON’T KNOW 8 • REFUSED 9 • *SR5b. How old were you at the time of this admission? • _________ YEARS OLD • DON’T KNOW 998 • REFUSED 999 • *SR5c. How much time did you stay in the hospital during this admission? • ____________ DURATION NUMBER 75% people have this design

8. *SR17. (IF *SR16 EQUALS ‘1’: Which ones? Just give me the letters. PROBE: Any other? / ALL OTHERS: (RB, PG 19) which of the following types of professionals did you ever see about problems with your emotions or nerves or your use of alcohol or drugs? Just give me the letters. (PROBE: Any others?) RECORD ALL MENTIONS • A. PSYCHIATRIST 1 • B. GENERAL PRACTITIONER OR FAMILY DOCTOR 2 • C. ANY OTHER MEDICAL DOCTOR, LIKE A CARDIOLOGIST OR (WOMEN: GYNECOLOGIST / MEN: UROLOGIST) 3 • D. PSYCHOLOGIST 4 • E. SOCIAL WORKER 5 • F. COUNSELOR 6 • G. ANY OTHER MENTAL HEALTH PROFESSIONAL, SUCH AS A PSYCHOTHERAPIST OR MENTAL HEALTH NURSE 7 • H. A NURSE, OCCUPATIONAL THERAPIST, OR OTHER HEALTH PROFESIONAL...……….8 • I. A RELIGIOUS OR SPIRITUAL ADVISOR LIKE A MINISTER, PRIEST, PASTOR, RABBI 9 • J. ANY OTHER HEALER, LIKE AN HERBALIST, DOCTOR OF ORIENTAL MEDICINE, CHIROPRACTOR, SPIRITUALIST 10 • K. DON’T KNOW 11 • L. REFUSED 12 25% people have this design

10. Embedded Experiment for Detecting Response Bias • Interviewees were randomly divided into two groups. 75% received the standard ordering, 25% received the new ordering. • Why split 75-25, rather than 50-50? • How much of a difference does the ordering make?

11. Compare the Old and New Design – Service Use

12. Compare the Old and New Design – Other Variables

13. Correcting/Reducing Response Biases via Multiple Imputation • Goal: Correcting/reducing the underreporting of service use in the 75% group by using the data from the 25%. • A Grand Challenge: Imputation ≠ Randomization DESIRE: Imputed rates for 75% match the observed rates from 25% for any potential subpopulation of interest. REALITY: Can only include a handful of covariates due to identifiability, computational, and time constraints. • Why impute at all? Can’t we just use 25% for analysis?

14. Basic Model Setup • I: Group indicator: 0 for new design, and 1 for old design. • y: self-reported service use: 0 for no service, 1 for had service; • ξs: true service use: 0 for no service, 1 for had service; • ξl: lying under the old design: 0 for lying and 1 for telling the truth. • Of interest is the distribution of ξs| y=0, I=1.

15. First Model For Imputation • Let’s assume • The likelihood function for one observation is where I=0 for 25% group, and I=1 for 75% group.

16. Multivariate Probit Model • We have 10 lifetime service use variables: • Associated with them, there are 10 lying indicators: • Introducing latent variable Z: is = I{zis > 0} and il = I{zil > 0}, where Z=(z1s,…, z10s, z1l,…,z10l) 11 > 0 is a 10×10 with diagonal elements all 1, and where x is a p×1, and B 20×p.

17. Covariates • Categorical variables: marital status, insurance status, working status, region in the country, ethnicity, immigration status, gender, psychiatric disorder diagnostics. • “Continuous” variables: logarithm of annual income, total number of psychiatric disorders, social status, age, k10 distress (psychiatric disorder related variable). • Some categorical variables are treated as “continuous” to reduce “dummy variables”, hence the number of covariates. • These variables are “negotiated” with psychologists.

18. Lifetime Service Use Results

19. Any Service Rates

20. High Order Interactions • Two way interaction: • High order interactions: • Failure to capture high order interactions will lead to biases in imputing maxi Xi , hence biases for the “any rates”. • Though arguably this “over-imputation” might be correcting for the false correlation induced by the interaction between memory decay and lying behavior, because it was not observed for the “last-12 month” imputation.

21. Model Improvement • The first model failed to capture the high-order interactions leading to the over-imputation of the “overall rates”. • Need hierarchical modeling using natural service type: Specialist, Generalist, Human Services, Alternative Services • We use  as the type indicator and  the service use within each type.

22. Second Model - Continuation Ratio Model • Let j1 and j2 follows multivariate probit model, that is, • z1={zj1} and z2={zj2} are independent and • A strong prior on  is imposed to avoid non-identifiability. b ~ N(2, 0.01),  ~ Inv-Wishart(I, 16) • When  →∞, this model reduces to the first model. • Cox (1972, JRSSB) proportional hazard model Heagerty and Zeger (2000, Biometrics) multivariate continuation ratio model.

23. Lifetime Service Use Results – Second Model

25. Imputation Quality Checking: stratified by variables included in the model

26. Imputation Quality Checking: stratified by variables not included in the model

27. Why the problem is so hard? – Stratified by insurance type (included in the model)

29. Effective sample size

30. The impact of the weights

31. The Failing of Randomization

32. Endless Future Work • Understand the CR model better via model diagnosis; • Further model improvement for subpopulations; • More efficient MCMCs for general applications; • General model strategy with many variables, small sample sizes, and potentially many subpopulations of interest. • “Data mining” tools for discovering problematic subpopulations(almost the same as discovering “bad genes”), even just for “buyers be aware” .

33. Disparities in Defining Disparity: Statistical Conceptual Frameworks • Joint work with • Naihua Duan1,2, Julia Lin3, Chih-nan Chen3, Margarita Alegria3,4 1 University of California, Los Angeles 2 Columbia University and New York State Psychiatric Institute 3Cambridge Health Alliance 4 Harvard Medical School

34. Ambiguities in IOM’s Definition • IOM (2003): Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare • IOM defines disparities in healthcare as “racial or ethnic differences in the quality of healthcare that are not due to access-related factors or clinical needs, preferences, and appropriateness of intervention.” • Explicit recognition of the role of causality.

35. Important Issues in IOM’s Definition of Healthcare Disparity • Difference is not necessarily disparity (which is not necessarily discrimination) • Justifiable (to be adjusted) vs. not justifiable (not to be adjusted) • Differences due to age, pre-existing health condition, etc., are OK • Difference due to literacy is not OK • Should access be adjusted for or not? • Grant Challenge: Causal relationship (“not due to”) Need to clearly spell out the causal relationships between “justifiable variables” and “not justifiable variables”

37. Rubin Causal Model • Causal model based on counterfactuals • Thought experiment, or actual experiment • What is changed • What is left unchanged • There is no single right or wrong thought experiment; different thought experiments should be developed for different clinical or policy goals

41. Statistical Conceptual Frameworks • Conceptual framework for conditional disparity, marginal disparity, and joint disparity • Define estimand: what do we want to estimate? • Estimation methods are important only after we understand what they intend to estimate.

42. Data Setting • Univariate Outcome Y: service use; number of visits. • Covariates for Adjustment X(A) : disorders; age; etc. • Covariates not for Adjustment X(N) : income; education; etc. • Deciding what to adjust is usually not a statistician’s task (though we can help), but how to adjust is.

43. Always Thinking Jointly … • Observed Data: • Key distributions: race-specific joint distribution. Race R, Y, X(A), X(N); and Weights W P(Y, X(A), X(N) | R) =P[Y|X(A), X(N), R]P[X(N)|X(A), R]P[X(A)|R] =P[Y|X(A), X(N), R]P[X(A)|X(N), R]P[X(N)|R]

44. Common “Unrealizable” Disparity

45. So what happens to the right eye? • Vision Acuity (AV) of left and right eyes are positively correlated in the population. • Suppose laser surgery is done to correct the AV of the left eye (L). What is going to happen to the AV of the right eye (R)? • We want to model P(R|L, S), where S=Surgery. • Will P(R|L, S)=P(R|L)? (e.g., L “causing” R) • Will P(R|L, S)=P(R)? (e.g., R “causing” L) • Orwill P(R|L, S) be something not estimable from the pre-surgery population? (and cross-sectional data …)

46. Conditional Disparity

47. Conditional Disparity: Assumed Causal Model X(N) Y R X(A) X(N): Not for adjust X(A): Adjust

48. Conditional Disparity: Thought Experiment X(N) Y R X(A)

49. Marginal Disparity

50. Marginal Disparity X(N) Y R X(A)