1 / 60

Epidemiologic Methods- Fall 2001

Epidemiologic Methods- Fall 2001. Definitions of Epidemiology. The study of the distribution and determinants (causes) of disease e.g. cardiovascular epidemiology The method used to conduct human subject research

rona
Download Presentation

Epidemiologic Methods- Fall 2001

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Epidemiologic Methods- Fall 2001

  2. Definitions of Epidemiology • The study of the distribution and determinants (causes) of disease • e.g. cardiovascular epidemiology • The method used to conduct human subject research • the methodologic foundation of any research where individual humans or groups of humans are the unit of observation

  3. Understanding Measurement: Aspects of Reproducibility and Validity • Review Measurement Scales • Reproducibility • importance • sources of measurement variability • methods of assessment • by variable type: interval vs categorical • Validity • methods of assessment • gold standards present • no gold standard available

  4. Clinical Research Sample Measure Analyze Infer

  5. A study can only be as good as the data . . . -Martin Bland

  6. Measurement Scales

  7. Reproducibility vs Validity • Reproducibility • the degree to which a measurement provides the same result each time it is performed on a given subject or specimen • Validity • from the Latin validus - strong • the degree to which a measurement truly measures (represents) what it purports to measure (represent)

  8. Reproducibility vs Validity • Reproducibility • aka: reliability, repeatability, precision, variability, dependability, consistency, stability • Validity • aka: accuracy

  9. Relationship Between Reproducibility and Validity Good Reproducibility Poor Validity Poor Reproducibility Good Validity

  10. Relationship Between Reproducibility and Validity Good Reproducibility Good Validity Poor Reproducibility Poor Validity

  11. Why Care About Reproducibility? Impact on Validity • Mathematically, the upper limit of a measurement’s validity is a function of its reproducibility • Consider a study to measure height in the community: • Assume the measurement has imperfect reproducibility: if we measure height twice on a given person, we get two different values; 1 of the 2 values must be wrong (imperfect validity) • If study measures everyone only once, errors, despite being random, lead to biased inferences in descriptive and analytic work (i.e. lack validity)

  12. Why Care About Reproducibility? Impact on Statistical Precision • Classical Measurement Theory: observed value (O) = true value (T) + measurement error (E) E is random and ~ N (0, 2E) When measuring a group of subjects, the variability of observed values is a combination of: the variability in their true values and measurement error 2O =2T + 2E

  13. Why Care About Reproducibility? 2O =2T + 2E • More measurement error means more variability in observed measurements • More variability of observed measurements has profound influences on statistical precision/power: • RCT’s: power to detect a treatment difference is reduced • Observational studies: power to detect an influence of a particular risk factor upon a given disease is reduced.

  14. Mathematical Definition of Reproducibility • Reproducibility • Varies from 0 (poor) to 1 (optimal) • As 2Eapproaches 0 (no error), reproducibility approaches 1

  15. Phillips and Smith, J Clin Epi 1993

  16. Sources of Measurement Variability • Observer • within-observer (intrarater) • between-observer (interrater) • Instrument • within-instrument • between-instrument • Subject • within-subject

  17. Sources of Measurement Variability • e.g. plasma HIV viral load • observer: measurement to measurement differences in tube filling, time before processing • instrument: run to run differences in reagent concentration, PCR cycle times, enzymatic efficiency • subject: biologic variation in viral load

  18. Assessing Reproducibility Depends on measurement scale • Interval Scale • within-subject standard deviation • coefficient of variation • Categorical Scale • Cohen’s Kappa

  19. Reproducibility of an Interval Scale Measurement: Peak Flow • Assessment requires >1 measurement per subject • Peak Flow Rate in 17 adults (Bland & Altman)

  20. Assessment by Simple Correlation

  21. Pearson Product-Moment Correlation Coefficient • r (rho) ranges from -1 to +1 • r • r describes the strength of the association • r2 = proportion of variance (variability) of one variable accounted for by the other variable

  22. r = 1.0 r = -1.0 r = 1.0 r = -1.0 r = 0.0 r = 0.8 r = 0.8 r = 0.0

  23. Correlation Coefficient for Peak Flow Data r ( meas.1, meas. 2) = 0.98

  24. Limitations of Simple Correlation for Assessment of Reproducibility • Depends upon range of data • e.g. Peak Flow • r (full range of data) = 0.98 • r (peak flow <450) = 0.97 • r (peak flow >450) = 0.94

  25. Limitations of Simple Correlation for Assessment of Reproducibility • Depends upon ordering of data • Measures linear association only

  26. 1700 1500 1300 1100 900 Meas. 2 700 500 300 100 100 300 500 700 900 1100 1300 1500 1700 Meas 1

  27. Limitations of Simple Correlation for Assessment of Reproducibility • Gives no meaningful parameter using the same scale as the original measurement

  28. Within-Subject Standard Deviation • Mean within-subject standard deviation (sw) = 15.3 l/min

  29. Computationally easier with ANOVA table: • Mean within-subject standard deviation (sw) :

  30. sw: Further Interpretation • If assume that replicate results: • arenormally distributed • mean of replicates estimates true value • sw estimates true standard deviation • 95% of replicates within (1.96)(sw) of true value x  true value sw (1.96) (sw)

  31. sw: Peak Flow Data • If assume that replicate results: • arenormally distributed • mean of replicates estimates true value • sw estimates true standard deviation • 95% of replicates within (1.96)(sw) = 30 l/min of true value x  true value sw = 15.3 l/min (1.96) (sw) = (1.96) (15.3) = 30

  32. sw: Further Interpretation • Difference between any 2 replicates for same person = diff = meas1 - meas2 • Because var(diff) = var(meas1) + var(meas2), therefore, s2diff = sw2 + sw2 = 2sw2 sdiff

  33. sw: Difference Between Two Replicates • If assume that differences: • arenormally distributed and mean of differences is 0 • sdiff estimates standard deviation • The difference between 2 measurements for the same subject is expected to be less than (1.96)(sdiff) = (1.96)(1.41)sw = 2.77sw for 95% of all pairs of measurements xdiff 0 sdiff (1.96) (sdiff)

  34. sw: Further Interpretation • For Peak Flow data: • The difference between 2 measurements for the same subject is expected to be less than 2.77sw =(2.77)(15.3) = 42.4 l/min for 95% of all pairs • Bland-Altman refer to this as the “repeatability” of the measurement

  35. One Common Underlying sw • Appropriate only if there is one sw • i.e, sw does not vary with true underlying value Kendall’s correlation coefficient = 0.17, p = 0.36 40 30 Within-Subject Std Deviation 20 10 0 100 300 500 700 Subject Mean Peak Flow

  36. Another Interval Scale Example • Salivary cotinine in children (Bland-Altman) • n = 20 participants measured twice

  37. Cotinine: Absolute Difference vs. Mean Kendall’s tau = 0.62, p = 0.001 4 3 Subject Absolute Difference 2 1 0 0 2 4 6 Subject Mean Cotinine

  38. Logarithmic Transformation

  39. Log Transformed: Absolute Difference vs. Mean Kendall’s tau=0.07, p=0.7 .6 .4 Subject abs log diff .2 0 -1 -.5 0 .5 1 Subject mean log cotinine

  40. sw for log-transformed cotinine data • sw • back-transforming to native scale: • antilog(sw) = antilog(0.175) = 10 0.175 = 1.49

  41. Coefficient of Variation • On the natural scale, there is not one common within-subject standard deviation for the cotinine data • Therefore, there is not one absolute number that can represent the difference any replicate is expected to be from the true value or from another replicate • Instead, = coefficient of variation

  42. Cotinine Data • Coefficient of variation = 1.49 -1 = 0.49 • At any level of cotinine, the within-subject standard deviation of repeated measures is 49% of the level

  43. Coefficient of Variation for Peak Flow Data • By definition, when the within-subject standard deviation is not proportional to the mean value, as in the Peak Flow data, then there is not a constant ratio between the within-subject standard deviation and the mean. • Therefore, there is not one common coefficient of variation • Estimating the the “average” coefficient of variation is not very meaningful

  44. Peak Flow Data: Use of Coefficient of Variation when sw is Constant

  45. Reproducibility of a Categorical Measurement: Chest X-Rays • On 2 different occasions, a radiologist is given the same 100 CXR’s from a group of high-risk smokers to evaluate for masses • How should reproducibility in reading be assessed?

More Related