1 / 42

Lecture 2 : Association and Inference

Lecture 2 : Association and Inference. Karen Bandeen -Roche, PhD Department of Biostatistics Johns Hopkins University. July 12, 2011. Introduction to Statistical Measurement and Modeling. Data motivation. Osteoporosis screening

amalie
Download Presentation

Lecture 2 : Association and Inference

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 2: Association and Inference Karen Bandeen-Roche, PhD Department of Biostatistics Johns Hopkins University July 12, 2011 Introduction to Statistical Measurement and Modeling

  2. Data motivation • Osteoporosis screening • Scientific question: Can we detect osteoporosis earlier and more safely? • Some related statistical questions: • Are ultrasound measurements associated with osteoporosis status? • How strong is the evidence of association? • By how much do the mean ultrasound values differ between those with and without osteoporosis? • How precisely can we determine that difference?

  3. Data motivation

  4. Outline • Association • Conditional probability • Joint distributions / Independence (note SRS) • Correlation • Statistical evidence / certainty • Confidence intervals • Statistical tests

  5. Association - Heuristic • Two random variables are associated if… • … they are “connected” or “related” • … knowing one helps predict the other • “Association” and “causality” are not equivalent • Two variables may be associated because a third variable causes each • Even if an association is causal the direction may not be clear (which causes which) • Association is necessary, but not sufficient, for causality

  6. Association - Formal • Tool: Conditional probability • Definition for two events A and B: If P(B) > 0, the conditional probability that event A occurs, given that event B has occurred, is P(A|B) := P(A∩B)/P(B) • Key concept: independence • A,B are pairwise independent if P{A,B} = P{A}P{B}. • Events {Aj, j = 1,...,n} are mutually independent if P{∩j=1n Aj} = ∏1n P{XjεAj} • A, B are independent iff P(A|B) = P(A) • Osteoporosis example: A=ultrasound<1750, B=case?

  7. Association - Formal • What about random variables? • Joint distribution function: FX,Y(x,y) = P{X≤x,Y≤y} = P{{X≤x} ∩ {Y≤y}} • Joint mass, density functions: • Discrete X, Y: pXY(x,y)=P{X=x,Y=y} • Continuous X, Y: fX,Y (x,y) = d2/(dsdt) FX,Y (s,t) |(x,y) • Conditional mass, density functions: • Discrete X, Y: pX|Y(x|y)=pXY(x,y)/pY(y) • Continuous X, Y: f(x|y) = fX,Y(x,y)/fY (y)

  8. Association - Formal • Key concept: independence • RVs X,Y are independent if for each x є SX , y є SY • FX,Y(x,y) = FX(x)FY(y) • pX|Y(x|y)=pX(x) (discrete X) • f(x|y) = fX(x) (continuous Y) • Osteoporosis example: X=ultrasound score, Y = 1 if osteoporosis case and Y=0 if osteoporosis control • Are the ultrasound densities the same for cases & controls? • Concepts generalize to mixed continuous / discrete (etc.), multiple random variables

  9. Association - Examples • Discrete random variables • Let X=1 if ultrasound < 1750, X=0 otherwise Y = 1 if osteoporosis case and Y=0 if control • Suppose the joint mass function in a population of older women is as follows: • Are low ultrasound and osteoporosis associated?

  10. Association - Examples • Continuous random variables • A pair of random variables (X,Y) is said to be bivariate normal if it is distributed according to the following density: • is the “correlation” parameter • Bivariate normal X,Y are independent if =0

  11. Association - Examples • Continuous random variables

  12. Association - Formal • A related concept to independence: Covariance • Heuristic: measures degree of linear relationship between variables. • Covariance=Cov(X,Y) =E{[X-E(X)][Y-E(Y)]} = E[XY] - E[X]E[Y] • Note: Details provided on adjunct handout • Two relationships now easy to characterize: a) Pairwise independence ⇒ Cov(X,Y) = 0;not vice versa b) Var(X+Y) = Var(X)+Var(Y)+2Cov(X,Y)

  13. Association - Example • Correlation of (ultrasound, DPA) = 0.36

  14. Association - Formal • X, Y are dependent (not independent) if there exist x1є SX, y1, y2 є SY such that • FX,Y(x1,y1) ≠ FX(x1)FY(y1) • pX|Y(x1|y1) ≠ pX(x1) ≠ pX|Y(x1|y2) (discrete X) • fX|Y(x1|y1) ≠ fX(x1) ≠ fX|Y(x1|y2)(continuous Y) • A sufficient but not necessary condition: X, Y are dependent (not independent) if the mean of X varies conditionally on Y E{X|Y} = ∫xfX|Y(x|Y)dx (continuous) = ΣxεSx xpX|Y(x|Y) (discrete) > Basis of regression!

  15. Data motivation • In our data sample the means are certainly different for the women with versus without osteoporosis • How persuasive is the evidence of a mean difference in a population of older women? = 1828 = 1688

  16. Basic paradigm of statistics • We wish to learn about populations • All about which we wish to make an inference • “True” experimental outcomes and their mechanisms • We do this by studying samples • Asubsetof a given population • “Represents” the population • Sample features are used to inferpopulation features • Method of obtaining the sample is important • Simple random sample: All population elements / outcomes have equal probability of inclusion

  17. Basic paradigm of statistics Probability Observed Value for a Representative Sample Truth for Population Our example: the difference in mean ultrasound measurement Between older women With and without osteoporosis Statistical inference

  18. Basic paradigm of statistics • Identify a parameter that summarizes the scientific quantity of interest • Obtain a representative sample of the target population • X1, … , Xn (possibly within groups) • Estimate the parameter using the sample (statistic) • Characterize the variability of the estimate • Make an inference, characterize its uncertainty, and draw a conclusion

  19. Basic paradigm • Identify a parameter that summarizes the scientific quantity of interest µ1 = E[X|Y=1] = mean ultrasound given osteoporosis µ0 = E[X|Y=0] = mean ultrasound given no osteoporosis Target parameter: µ1 - µ0 • Obtain a representative sample of the target population • The current sample was a clinical series. Questionable how representative.

  20. Basic paradigm • Estimate the parameter using the sample (statistic) • Many estimation methods have been developed. • Proposed estimator here: Means based on ECDF • “Method of moments” • Pause: Is this a good estimator? What makes it one? • Consider a generic estimator (statistic) gn(X) • Denote the target parameter we wish to estimate by Θ

  21. Basic paradigm • Some properties of a good estimator gn(X) • Accuracy with respect to target parameter Θ • Unbiased: E[gn(X) ] = Θ • That is, bias = E[gn(X)] – Θ= 0 • Consistent: For each ε > 0, limn→∞ P{|gn(X)-Θ|> ε} = 0or (stronger) P{limn→∞ gn(X) = Θ} = 1 • Precision: small Var(gn(X)) • Good tradeoff of accuracy and precision: • Low mean squared error = E(gn(X)-Θ)2 = Var(gn (X))+[Bias(gn (X))]2

  22. Basic paradigm • Estimate the parameter using the sample (statistic) • Characterize the variability of the estimate • Method 1: Calculate the variance directly • Assumption: Sampling method ⇒ mutual independence • Then, Var( ) = Var( ) + Var( ) • Var( ) = 1/n1 Var(X1) = σ12/n1 • Var( ) = σ12/n1 + σ02/n0

  23. *** Basic paradigm *** • Characterize the variability of the estimate • Method 1: Var( ) = σ12/n1 + σ02/n0 • What does this mean? • We “have” one possible sample among many that could have been taken from the population • Suppose it was drawn randomly • Definition: The values that the statistic of interest could have taken over all possible samples (of the same size randomly drawn from the population), and their probability measure, is called the sampling distributionof that statistic. • σ12/n1 + σ02/n0 is the variance of the sampling distribution • Important term: The standard deviation of the sampling distribution is called the standard error

  24. Way to “see” the sampling distribution: Bootstrap-Efron, 1979 • Idea: mimic sampling that produced the original sample. • Treat the sample as if it is the whole population. The original sample statistic becomes the true value (“truth”) we seek. • Draw the first bootstrap sample at random (with replacement) from the original sample and calculate the statistic of interest. • Repeat this process 1000* times. The distribution of bootstrapped statistics approximates the sampling distribution of the statistic. Efron, B. Bootstrap Methods: Another Look at the Jackknife. Ann Stat 1979; 7:1-26.

  25. Repeat 1000 Times Bootstrapped mean difference: -150.14 -118.16 -128.30 -168.46 -157.69 . . . -196.65 Original sample value of -140.48

  26. Characterize variability of gn(X) • Method 2: Var( ) estimated by the variance of the bootstrapped distribution • Variance is still a bit difficult to interpret • Method 3: Employ the Central Limit Theorem • Distributions of sample means converge to normal as n→∞ • Definition: Fgn(X)(s) converges to F(s) in distribution ⇒ limn→∞ P{gn(X)≤s} = F(s) at every continuity point of F. • Central limit theorem: Let X1,...,Xn be a sequence of mutually independent RVs with common distribution. Define Sn=Σi Xi. Then limn→∞ P{(Sn-nμ)/(σn1/2) ≤ z} = Φ(z) for every fixed z ,where Φ is the Normal CDF with mean=0 and variance=1.

  27. Characterize variability of gn(X) • Implications of the Central Limit Theorem • ~ 95% of estimates within +/- 1.96 standard errors of µ1 - µ0

  28. Characterize variability of gn(X) • Implications of the Central Limit Theorem • ~ 95% of estimates within +/- 1.96 standard errors of µ1 - µ0 • P{µ1 - µ0 ε ± 1.96 SE( )} = 0.95 • Many gn(X) converge in distribution to normal • Broadly: P[Θεgn(X) ± z(1-α/2) SE{gn(X)}] ≈ 1-α with z(1-α/2) = Q{(1-α/2)} of the normal distribution with µ=0 and σ=1 • An interval I(X) = [L{gn(X)},U{gn(X)}] satisfying P{Θε I(X)}= 1-α is called a 100x(1-α) confidence interval

  29. Bootstrap Confidence Interval Original sample value of -140.48 Fact: the interval that includes the middle 95% of the bootstrapped statistics covers the true unknown population value for roughly 95% of samples if the sampling distribution of the statistic or a function thereof is roughly Gaussian.

  30. Analytic Confidence Interval (CI) • Sample mean difference = 1688-1828 = -140 • Var( ) = σ12/n1 + σ02/n0; SE = square root of this • Estimate σ12, σ02 by sample analogss12, s02 = (5362,13570) • n1 = n0 = 21 • SE = √{5362/21 + 13570/21} = 30.03 • 95% CI = (-140-1.96*30.03,-140+1.96*30.03)= (-199,-81) = an interval in which we have 95% confidence for including the difference in mean ultrasound scores between osteoporotic and non-osteoporotic older women

  31. Analytic CI – A detail • The preceding calculation assumed / √(s12/n1 +s02/n0) approximately distributed as normal mean=0, variance=1 • This does not account for variability in substituting s for σ • Rather, the correct distribution is “t” (close to normal for large n) • With this correction the 95% CI is (-202,-79)

  32. Basic paradigm of statistics • Make an inference, characterize its uncertainty, and draw a conclusion • Inference Method 1: (-202,-79) is a 95% CI for the difference in mean ultrasound scores between those with, without osteoporosis. • Conclusion: Based on the data, we are confident that the mean ultrasound score is substantially lower in osteoporotic older women than in those without osteoporosis. • The conclusion is a population statement.

  33. Basic paradigm of statistics • Make an inference, characterize its uncertainty, and draw a conclusion • Inference Method 2: Statistical testing • Two goals • Assess evidence for a statement / hypothesis Ultrasound measurements are (or are not) associated with osteoporosis status • Make yes/no decision about question of interest: Are ultrasound measurements associated with osteoporosis status?

  34. Decision making • Neyman-Pearson framework for hypothesis testing • Define complementary hypotheses about the truth in the population • Denote as θ the parameter space={possible values of Θ} • Define θ0, θ1 as distinct subsets of θ corresponding to the different hypotheses • Example: No difference in means versus a difference in means • “Null” Hypothesis - H0: Θ ε θ0 (H0: µ1 - µ0 = 0) • “Alternative” Hypothesis - H1: Θ ε θ1 (H1: µ1 - µ0 ≠ 0)

  35. Decision making • Choose an estimator of Θ, gn(X) (as above) • Main idea: If the value of gn(X) observed in one’s data is “unusual” assuming H0, then conclude H0 is not a reasonable model for the data and decide against it • Testing – steps: • Compute distribution of gn(X) for Θ = θ0 • Define a “rejection” region of values far from θ0occurring with low probability = αwhen H0 is true • Reject H0 if gn(x) is in the “rejection” region; do not reject it otherwise.

  36. Rejection region - example • We are interested in • If H0 is true (µ1 - µ0 = 0), then ( - 0)/SE is approximately distributed as normal with mean = 0 and variance = 1 (henceforth, N(0,1) ) Density if H0 is true

  37. Possible Results of Hypothesis Testing Reject when H0 true Fail to reject when H1 true

  38. Significance testing • Variant of Neyman-Pearson • Based on calculation of a p-value: The probability of observing a test statistic as or more extreme than occurred in sample under the null hypothesis • As or more extreme: as different or more different from the hypothesized value • p-value sometimes used as measure of evidence against null, and often, "for" alternative

  39. Osteoporosis data • Are ultrasound measurements associated with osteoporosis status? • Hypotheses: H0: µ1 - µ0 = 0 vs.H1: µ1 - µ0 ≠ 0 • Test statistic: /√(s12/n1 +s02/n0) =140/30.03=4.66 • Rejection region: > 1.96 or < -1.96 • 4.66 > 1.96, thus we reject H0(again, approximate: “t”) • p-value: probability of a value as far or farther from 0 than 4.66 if the true mean difference were 0 • P{gn(X) > 4.66 or gn(X) < -4.66}= 3.16E-06 • Conclusion: Data strongly support a “yes” answer.

  40. Data motivation • Osteoporosis screening • Scientific question: Can we detect osteoporosis earlier and more safely? • Some related statistical questions: • Are ultrasound measurements associated with osteoporosis status? - Testing indicates “yes”. • How strong is the evidence of association? Strong (95% CI very substantially excludes values near 0) • By how much do the mean ultrasound values differ between those with and without osteoporosis? We estimate that older women with osteoporosis have mean 140dB/MHz lower than those without osteoporosis • How precisely can we determine that difference? Standard error = 30.03; 95% CI = (-202,-79)

  41. Main points • Variables (characteristics, features, etc.) are associated if they are statistically dependent. • Covariance / correlation measure linear association • We use representative samples to study features of populations • Estimators (“statistics”) provide approximations of population parameters • Possible values and their probabilities are characterized by sampling distributions • Standard errors and confidence intervals summarize their associated variability / uncertainty

  42. Main points • Statistical tests evaluate evidence and inform decisions about scientific (and other) questions • Neyman-Pearson paradigm • Hypothesis testing • Significance testing

More Related