1 / 63

Epidemiologic Methods

Epidemiologic Methods. Definitions of Epidemiology. The study of the distribution and determinants (causes) of disease e.g. cardiovascular epidemiology The method used to conduct human subject research

seda
Download Presentation

Epidemiologic Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Epidemiologic Methods

  2. Definitions of Epidemiology • The study of the distribution and determinants (causes) of disease • e.g. cardiovascular epidemiology • The method used to conduct human subject research • the methodologic foundation of any research where individual humans or groups of humans are the unit of observation

  3. Understanding Measurement: Aspects of Reproducibility and Validity • Review Measurement Scales • Reproducibility • importance • methods of assessment • by variable type: interval vs categorical • intra- vs. inter-observer comparison • Validity • methods of assessment • gold standards present • no gold standard available

  4. Clinical Research Sample Measure Analyze Infer

  5. A study can only be as good as the data . . . -Martin Bland

  6. Measurement Scales

  7. Reproducibility vs Validity • Reproducibility • the degree to which a measurement provides the same result each time it is performed on a given subject or specimen • Validity • from the Latin validus - strong • the degree to which a measurement truly measures (represents) what it purports to measure (represent)

  8. Reproducibility vs Validity • Reproducibility • aka: reliability, repeatability, precision, variability, dependability, consistency, stability • Validity • aka: accuracy

  9. Relationship Between Reproducibility and Validity Good Reproducibility Poor Validity Poor Reproducibility Good Validity

  10. Relationship Between Reproducibility and Validity Good Reproducibility Good Validity Poor Reproducibility Poor Validity

  11. Why Care About Reproducibility? Impact on Validity • Mathematically, the upper limit of a measurement’s validity is a function of its reproducibility • Consider a study to measure height in the community: • if we measure height twice on a given person and get two different values, then one of the two values must be wrong (invalid) • if study measures everyone only once, errors, despite being random, may not balance out • final inferences are likely to be wrong (invalid)

  12. Why Care About Reproducibility? Impact on Statistical Precision • Classical Measurement Theory: observed value (O) = true value (T) + measurement error (E) E is random and ~ N (0, 2E) Therefore, when measuring a group of subjects, the variability of observed values is a combination of: the variability in their true values and measurement error 2O =2T + 2E

  13. Why Care About Reproducibility? 2O =2T + 2E • More measurement error means more variability in observed measurements • More variability of observed measurements has profound influences on statistical precision/power: • Descriptive study: less precise estimates of given traits • RCT’s: power to detect a treatment difference is reduced • Observational studies: power to detect an influence of a particular exposure upon a given outcome is reduced.

  14. Conceptual Definition of Reproducibility • Reproducibility • Varies from 0 (poor) to 1 (optimal) • As 2Eapproaches 0 (no error), reproducibility approaches 1

  15. Phillips and Smith, J Clin Epi 1993

  16. Sources of Measurement Variability • Observer • within-observer (intrarater) • between-observer (interrater) • Instrument • within-instrument • between-instrument • Subject • within-subject

  17. Sources of Measurement Variability • e.g. plasma HIV viral load • observer: measurement to measurement differences in tube filling, time before processing • instrument: run to run differences in reagent concentration, PCR cycle times, enzymatic efficiency • subject: biologic variation in viral load

  18. Assessing Reproducibility Depends on measurement scale • Interval Scale • within-subject standard deviation • coefficient of variation • Categorical Scale • Cohen’s Kappa

  19. Reproducibility of an Interval Scale Measurement: Peak Flow • Assessment requires >1 measurement per subject • Peak Flow Rate in 17 adults (Bland & Altman)

  20. Assessment by Simple Correlation

  21. Pearson Product-Moment Correlation Coefficient • r (rho) ranges from -1 to +1 • r • r describes the strength of the association • r2 = proportion of variance (variability) of one variable accounted for by the other variable

  22. r = 1.0 r = -1.0 r = 1.0 r = -1.0 r = 0.0 r = 0.8 r = 0.8 r = 0.0

  23. Correlation Coefficient for Peak Flow Data r ( meas.1, meas. 2) = 0.98

  24. Limitations of Simple Correlation for Assessment of Reproducibility • Depends upon range of data • e.g. Peak Flow • r (full range of data) = 0.98 • r (peak flow <450) = 0.97 • r (peak flow >450) = 0.94

  25. Limitations of Simple Correlation for Assessment of Reproducibility • Depends upon ordering of data • Measures linear association only

  26. 1700 1500 1300 1100 900 Meas. 2 700 500 300 100 100 300 500 700 900 1100 1300 1500 1700 Meas 1

  27. Limitations of Simple Correlation for Assessment of Reproducibility • Gives no meaningful parameter for the issue

  28. Within-Subject Standard Deviation • Mean within-subject standard deviation (sw) = 15.3 l/min

  29. Computationally easier with ANOVA table: • Mean within-subject standard deviation (sw) :

  30. sw: Further Interpretation • If assume that replicate results: • normally distributed • mean of replicates estimates true value • standard deviation estimated by sw • Then 95% of replicates will be within (1.96)(sw) of the true value • For Peak Flow data: • 95% of replicates will be within (1.96)(15.3) = 30.0 l/min of the true value

  31. sw: Further Interpretation • Difference between any 2 replicates for same person = diff = meas1 - meas2 • Because var(diff) = var(meas1) + var(meas2), therefore, s2diff = sw2 + sw2 = 2sw2 sdiff • If assume the distribution of the differences between pairs is N(0, 2diff), therefore, • The difference between 2 measurements for the same subject is expected to be less than (1.96)(sdiff) = (1.96)(1.41)sw = 2.77sw for 95% of all pairs of measurements

  32. sw: Further Interpretation • For Peak Flow data: • The difference between 2 measurements for the same subject is expected to be less than 2.77sw =(2.77)(15.3) = 42.4 l/min for 95% of all pairs • Bland-Altman refer to this as the “repeatability” of the measurement

  33. Interpreting sw • Appropriate only if there is one sw • if sw does not vary with the true underlying value Kendall’s correlation coefficient = 0.17, p = 0.36 40 30 Within-Subject Std Deviation 20 10 0 100 300 500 700 Subject Mean Peak Flow

  34. Another Interval Scale Example • Salivary cotinine in children (Bland-Altman) • n = 20 participants measured twice

  35. Simple Correlation of Two Trials 6 4 trial 1 2 0 0 2 4 6 trial 2

  36. Correlation of Cotinine Replicates

  37. Cotinine: Absolute Difference vs. Mean Kendall’s tau = 0.62, p = 0.001 4 3 Subject Absolute Difference 2 1 0 0 2 4 6 Subject Mean Cotinine

  38. Logarithmic Transformation

  39. Log Transformed: Absolute Difference vs. Mean Kendall’s tau=0.07, p=0.7 .6 .4 Subject abs log diff .2 0 -1 -.5 0 .5 1 Subject mean log cotinine

  40. sw for log-transformed cotinine data • sw • back-transforming to original units: • antilog(sw) = antilog(0.175) = 1.49

  41. Coefficient of Variation • On the natural scale, there is not one common within-subject standard deviation for the cotinine data • Therefore, there is not one absolute number that can represent the difference any replicate is expected to be from the true value or from another replicate • Instead, = coefficient of variation

  42. Cotinine Data • Coefficient of variation = 1.49 -1 = 0.49 • At any level of cotinine, the within-subject standard deviation of repeated measures is 49% of the level

  43. Coefficient of Variation for Peak Flow Data • By definition, when the within-subject standard deviation is not proportional to the mean value, as in the Peak Flow data, then there is not a constant ratio between the within-subject standard deviation and the mean. • Therefore, there is not one common coefficient of variation • Estimating the coefficient of variation by taking the common within-subject standard deviation and dividing by the overall mean of the subjects is not very meaningful

  44. Intraclass Correlation Coefficient, rI • rI • Averages correlation across all possible ordering of replicates • Varies from 0 (poor) to 1 (optimal) • As 2E approaches 0 (no error), rI approaches 1 • Advantages: not dependent upon ordering of replicates; does not mistake linear association for agreement; allows >2 replicates • Disadvantages: still dependent upon range of data in sample, still does not give a meaningful parameter on the actual scale of measurement in question

  45. Intraclass Correlation Coefficient, rI • rI • where: • m = no. of replicates per person • SSb = sum of squares between subjects • SSt = total sum of squares • rI(peak flow) = 0.98 • rI(cotinine) = 0.69

  46. Reproducibility of a Categorical Measurement: Chest X-Rays • On 2 different occasions, a radiologist is given the same 100 CXR’s from a group of high-risk smokers to evaluate for masses • How should reproducibility in reading be assessed?

More Related