Tests for Binary/Categorical outcomes

Tests for Binary/Categorical outcomes

Binary or categorical outcomes (proportions)

Chi-square test From an RCT of probiotic supplementation during pregnancy to prevent eczema in the infant: Table 3. Cumulative incidence of eczema at 12 months of age Kim et al. Effect of probiotic mix (Bifidobacterium bifidum, Bifidobacterium lactis, Lactobacillus acidophilus) in the primary prevention of eczema: a double-blind, randomized, placebo-controlled trial. Pediatric Allergy and Immunology. Published online October 2009.

Chi-square test Statistical question: Does the proportion of infants with eczema differ in the treatment and control groups? • What is the outcome variable? Eczema in the first year of life (yes/no) • What type of variable is it? Binary • Are the observations correlated? No • Are groups being compared and, if so, how many? Yes, two groups • Are any of the counts smaller than 5? No, smallest is 12 (probiotics group with eczema)  chi-square test or relative risks, or both

Chi-square test of Independence Chi-square test allows you to compare proportions between 2 or more groups (ANOVA for means; chi-square for proportions).

Example 2 • Asch, S.E. (1955). Opinions and social pressure. Scientific American, 193, 31-35.

The Experiment • A Subject volunteers to participate in a “visual perception study.” • Everyone else in the room is actually a conspirator in the study (unbeknownst to the Subject). • The “experimenter” reveals a pair of cards…

The Task Cards Standard line Comparison lines A, B, and C

The Experiment • Everyone goes around the room and says which comparison line (A, B, or C) is correct; the true Subject always answers last – after hearing all the others’ answers. • The first few times, the 7 “conspirators” give the correct answer. • Then, they start purposely giving the (obviously) wrong answer. • 75% of Subjects tested went along with the group’s consensus at least once.

Further Results • In a further experiment, group size (number of conspirators) was altered from 2-10. • Does the group size alter the proportion of subjects who conform?

Conformed? Number of group members? 2 4 6 8 10 Yes 20 50 75 60 30 No 80 50 25 40 70 The Chi-Square test Apparently, conformity less likely when less or more group members…

20 + 50 + 75 + 60 + 30 = 235 conformed • out of 500 experiments. • Overall likelihood of conforming = 235/500 = .47

Conformed? Number of group members? 2 4 6 8 10 Yes 47 47 47 47 47 No 53 53 53 53 53 Expected frequencies if no association between group size and conformity…

Do observed and expected differ more than expected due to chance?

Degrees of freedom = (rows-1)*(columns-1)=(2-1)*(5-1)=4 Chi-Square test

Degrees of freedom = (rows-1)*(columns-1)=(2-1)*(5-1)=4 Chi-Square test Rule of thumb: if the chi-square statistic is much greater than it’s degrees of freedom, indicates statistical significance. Here 85>>4.

Interpretation • Group size and conformity are not independent, for at least some categories of group size • The proportion who conform differs between at least two categories of group size • Global test (like ANOVA) doesn’t tell you which categories of group size differ

Caveat **When the sample size is very small in any cell (<5), Fisher’s exact test is used as an alternative to the chi-square test.

Review Question 1 I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare years of schooling (a normally distributed variable) between the three groups. What test should I use? • Repeated-measures ANOVA. • One-way ANOVA. • Difference in proportions test. • Paired ttest. • Chi-square test.

Review Question 2 I divide my study population into smokers, ex-smokers, and never-smokers; I want to compare the proportions of each group that went to graduate school. What test should I use? • Repeated-measures ANOVA. • One-way ANOVA. • Difference in proportions test. • Paired ttest. • Chi-square test.

Risk ratios and odds ratios From an RCT of probiotic supplementation during pregnancy to prevent eczema in the infant: Table 3. Cumulative incidence of eczema at 12 months of age Kim et al. Effect of probiotic mix (Bifidobacterium bifidum, Bifidobacterium lactis, Lactobacillus acidophilus) in the primary prevention of eczema: a double-blind, randomized, placebo-controlled trial. Pediatric Allergy and Immunology. Published online October 2009.

Treatment Placebo + 12 22 - 21 13 Treatment Group Eczema Corresponding 2x2 table

Risk ratios and odds ratios Statistical question: Does the proportion of infants with eczema differ in the treatment and control groups? • What is the outcome variable? Eczema in the first year of life (yes/no) • What type of variable is it? Binary • Are the observations correlated? No • Are groups being compared and, if so, how many? Yes, binary • Are any of the counts smaller than 5? No, smallest is 12 (probiotics group with eczema)  chi-square test or relative risks, or both

Odds vs. Risk (=probability) 1:1 3:1 1:9 1:99 Note: An odds is always higher than its corresponding probability, unless the probability is 100%.

Risk ratios and odds ratios • Absolute risk difference in eczema between treatment and placebo: 36.4%-62.9%=-26.5% (p=.029, chi-square test). • Risk ratio: • Corresponding odds ratio: There is a 26.5% decrease in absolute risk, a 42% decrease in relative risk, and a 66% decrease in relative odds.

Why do we ever use an odds ratio?? • We cannot calculate a risk ratio from a case-control study (since we cannot calculate the risk of developing the disease in either exposure group). • The multivariate regression model for binary outcomes (logistic regression) gives odds ratios, not risk ratios. • The odds ratio is a good approximation of the risk ratio when the disease/outcome is rare (~<10% of the control group)

Interpretation of the odds ratio: • The odds ratio will always be bigger than the corresponding risk ratio if RR >1 and smaller if RR <1 (the harmful or protective effect always appears larger) • The magnitude of the inflation depends on the prevalence of the disease.

1 1 When a disease is rare: P(~D) = 1 - P(D)  1 The rare disease assumption

Odds ratio Odds ratio Odds ratio Risk ratio Risk ratio Odds ratio Risk ratio Risk ratio The odds ratio vs. the risk ratio Rare Outcome 1.0 (null) Common Outcome 1.0 (null)

General Rule of Thumb: “OR is a good approximation as long as the probability of the outcome in the unexposed is less than 10%” When is the OR is a good approximation of the RR?

Recall… • Split-face trial: • Researchers assigned 56 subjects to apply SPF 85 sunscreen to one side of their faces and SPF 50 to the other prior to engaging in 5 hours of outdoor sports during mid-day. • Sides of the face were randomly assigned; subjects were blinded to SPF strength. • Outcome: sunburn Russak JE et al. JAAD 2010; 62: 348-349.

Results: Table I -- Dermatologist grading of sunburn after an average of 5 hours of skiing/snowboarding (P = .03; Fisher’s exact test) The authors use Fisher’s exact test to compare 1/56 versus 8/56. But this counts individuals twice and ignores the correlations in the data!

McNemar’s test Statistical question: Is SPF 85 more effective than SPF 50 at preventing sunburn? • What is the outcome variable? Sunburn on half a face (yes/no) • What type of variable is it? Binary • Are the observations correlated? Yes, split-face trial • Are groups being compared and, if so, how many? Yes, two groups (SPF 85 and SPF 50) • Are any of the counts smaller than 5? Yes, smallest is 0  McNemar’s test exact test (if bigger numbers, would use McNemar’s chi-square test)

Correct analysis of data… Table 1. Correct presentation of the data from: Russak JE et al. JAAD 2010; 62: 348-349. (P = .016; McNemar’s test). Only the 7 discordant pairs provide useful information for the analysis!

McNemar’s exact test… • There are 7 discordant pairs; under the null hypothesis of no difference between sunscreens, the chance that the sunburn appears on the SPF 85 side is 50%. • In other words, we have a binomial distribution with N=7 and p=.5. • What’s the probability of getting X=0 from a binomial of N=7, p=.5? • Probability = • Two-sided probability =

McNemar’s chi-square test • Basically the same as McNemar’s exact test but approximates the binomial distribution with a normal distribution (works well as long as sample sizes in each cell >=5)

Recall: Political party and drinking… Drinking by political affiliation

Recall: Political party and alcohol… This association could be analyzed by a ttest or a linear regression or also by logistic regression: Republican (yes/no) becomes the binary outcome. Alcohol (continuous) becomes the predictor.

Logistic regression • Statistical question: Does alcohol drinking predict political party? • What is the outcome variable? Political party • What type of variable is it? Binary • Are the observations correlated? No • Are groups being compared? No, our independent variable is continuous •  logistic regression

Logit function =log odds of the outcome The logistic model… ln(p/1- p) =  + 1*X

Linear function of risk factors for individual i: 1x1+ 2x2 + 3x3+ 4x4 … Baseline odds Logit function (log odds) The Logit Model (multivariate)

Review question 7 • If X=.50, what is the logit (=log odds) of X? • .50 • 0 • 1.0 • 2.0 • -.50

Tests for Binary/Categorical outcomes