LEVELS OF EVIDENCE FROM DIABETES REGISTRIES Registry-based Epidemiology?

1. LEVELS OF EVIDENCE FROM DIABETES REGISTRIES Registry-based Epidemiology? John M. Lachin Professor of Biostatistics, Epidemiology and Statistics The Biostatistics Center The George Washington University

2. EuBIRO-D vs. USA Ciao Fabrizio e Massimo No regional or national healthcare program No national or regional registries HMO network Translating Research into Action for Diabetes Comparative Effectiveness Research Agency for Healthcare Quality and Research: patient satisfaction, quality of life National Institutes of Health: Clinical outcomes GRADE study

3. Science and Uncertainty Jacob Bronowsky: All information is imperfect. We have to treat it with humility... Errors are inextricably bound up with the nature of human knowledge… The degree of uncertainty is controlled through the application of the scientific method, and is quantified through statistics.

4. Statistical Test of an Hypothesis Null Hypothesis (H0): The hypothesis to be disproven The hypothesis of no difference. Alternative Hypothesis (H1): The hypothesis to be proven The hypothesis that a difference exists. Two types of errors: Type I: False positive, probability  Type II: False negative, probability  Power = 1 - 

5. Factors that Affect  and Power Selection and Observational/Experimental Bias Poor study design or execution Missing data Reproducibility (precision) of assessments

6. Missing DataThe Fundamental Issue - BIAS Numerators and denominators may be biased Estimates of population parameters, differences between treatments or exposure groups may be biased. Statistical analyses, p–values and confidence limits may be biased. p = 0.05 may mean a false positive error rate () much greater than 0.05; N=800, 20% missing in treated/exposed, true  ≈ 0.40.

7. Can’t Statistics Handle This? Not definitively. The magnitude of the bias can not be estimated, no correction possible. Analyses can be conducted under certain assumptions. But there is no way to prove that the assumptions apply. Best way to deal with missing data is to prevent it.

8. Sample Size Adjustments Can adjust sample size to allow for losses-to-follow-up and missing data, e.g. increase N by 10% if expect 10% losses BUT, this adjusts only for the loss of information, NOT for any bias introduced by missing data.

9. Precision or Reliability of Measures Reliability coefficient  = proportion of total variation between subjects due to variation in the true values. 1 -  = proportion of variation due to random errors of collection, processing and measurement.

10. Impact of Reliability Power decreases as  decreases. Power Reliability ()

11. Impact of Reliability If N is the sample size needed for a precise measure then N/ is needed for an imprecise measure.

12. Impact of Reliability Maximum possible correlation between Y and X is a function of the respective reliabilities: Max(R2) = x y

13. Impact of Misclassifications m = fraction of treatment or exposure misclassifications, or fraction of outcomes misclassified N/(1-2m)2 is needed

14. Randomized Clinical Trial Randomization: Subjects assigned to each treatment independently of patient characteristics No selection bias. Treatment groups expected to be similar for all variables measured and unmeasured. No confounding of the experimental treatment with other uncontrolled factors May infer a cause – effect relationship between treatment and the outcome, provided the trial is of good quality.

15. Randomized Clinical Trial Precisely defined population Precisely defined exposure (the treatments) Precisely defined outcome measure Results clearly interpretable

16. Observational Study Many types, e.g. case-control study Prospective cohort study No randomized controls Maybe a precisely defined population Maybe a precisely defined exposure (the treatments) Maybe a precisely defined outcome measure

17. Observational Study Many potential biases Selection bias – composition of groups Confounding with other factors Statistical adjustments substituted for randomization

18. Observational Study Necessary in settings where a randomized study is impossible Smoking and lung cancer Generally describe an association between the exposure factor and an outcome that may not represent a causal relationship. Difficult to establish causality, though possible with replication of a highly specific association.

19. Observational Evidence The essential issues with observational evidence is the degree to which an observed relationship can or can not be explained by other variables, other mechanisms, or biases – even after statistical adjustment

20. Confounding When the study factor (groups) are associated with another (confounding) factor that is a direct cause of the outcome. Coffee consumption and cancer. Coffee consumption confounded with smoking. Higher fraction of smokers among coffee drinkers.

21. Statistical Adjustment for Confounding Regression or stratification model including the study factor and the possible confounding factor(s) Assumes that the operating confounding factors have been identified and measured. Assumes that the regression model specifications are correct.

22. Statistical Adjustment for Confounding Estimates the association of the factor with the outcome IF the confounding factor were equally distributed among the groups. Difference in cancer risk between coffee drinkers and non-drinkers IF the fraction of smokers was the same among drinkers and non-drinkers. Coffee drinking and smoking are alterable. Thus, the results would have a population interpretation.

23. Statistical Adjustments NOT all covariate imbalances introduce bias, in which case adjustment itself introduces bias. Gender inherently confounded with body weight Gender adjusted for body weight estimates the gender difference if males and females had the same weight distribution.

24. Statistical Adjustments Adjustment for weight provides a biased estimate of the overall male:female difference in risk in the population But weight-adjusted estimate describes the additionalmale:female difference in risk, if any, that is associated with gender differences other than weight Of mechanistic interest.

25. Omitted Covariates Observational study can only adjust for what has been measured. Adjustment for observed factors can not eliminate bias due to imbalances in unmeasured covariates.

26. Inappropriate Covariates Analysis should follow the prospective history of covariates Statistically invalid to define a covariate over a period of exposure that goes beyond the observation of an event. Example, mean HbA1c over 5 years as a predictor of outcomes observed during the 5 years. Rather, use the mean HbA1c up to the time of each successive event.

27. Confounding by Indication In some cases, however, exposure to a factor (e.g. drug) may be confounded with the indications leading to the exposure. Example: statins indicated in the presence of hyperlipidemia. Recent data suggests that statin use may also increase risk of T2D in IFG/IGT. But is the increased risk due to the statin use or the prior history of hyperlipidemia?

28. Confounding by Indication In other cases an adjusting factor (e.g. dose) may likewise be confounded with an indication. Example: Hemkens et al. analysis of the association of insulin glargine vs. human insulin with cancer in a German claims database. 14% decrease in age, gender adjusted risk. But substantial dose imbalance. 14% increase in risk when also adjusted for dose.

29. Reasons for Dose Imbalance Confounding by indication, or allocation bias. High or low glargine (or human insulin) dose may be determined by unmeasured patient factors that are differentially distributed within groups. e.g. high glargine dose only administered to severely ill patients. Impossible to statistically adjust for such confounding Adjusted analysis results are biased.

30. Registries Many types: 100% population captured, e.g. public health care system Non-random subsample, e.g. insurance provider or hospital based In latter case, registry population may not represent the full population of interest Inherently prospective But no standardized follow-up schedule

31. Registries Relies on data capture in conjunction with the administration of medical care No specific exposure of interest when established, in epidemiological sense No specific outcome measure of interest. Rather medical status and treatment recorded (possible exposures) and other major morbidities and mortality recorded (possible outcomes).

32. Registries Epidemiologic analyses may be attempted. But, difficult to precisely define exposure to a factor: When is a subject First at risk of being exposed (e.g. when is a drug introduced to the market?) Actually first exposed (e.g. starts drug) Removed from exposure (e.g. off drug) Confounding by indication often an issue

33. Registries Coding, classification of events may not be standardized Often no adjudication May be difficult to determine whether or exactly when an outcome event occurred, e.g. macroalbuminuria is “interval-censored” May be difficult to determine when subject no longer at risk (right censored) Incidence may be difficult to assess reliably.

34. Registries - Uses Prevalence Distribution of patient status or conditions in the population Cross-sectional associations If “representative” but not proportionally, weighted analyses can provide estimates in the broader population. Disadvantaged populations (poverty, uninsured) may not be represented

35. Registries - Epidemiology Exposure to a factor and outcomes Open to many biases. Statistical adjustments may be inadequate. But, a registry can be the foundation for first-rate epidemiologic studies.

36. Registries - Epidemiology Nested case-control studies Sub-sample of possible cases that is carefully adjudicated Sub-sample of possible controls (matched by follow-up time) also verified. Exposure (risk) and confounding factors also verified.

37. Registries - Epidemiology Prospective cohort studies Identify eligible subjects -- representative of the registry (general) population Formally enroll subjects (consent) with a systematic follow-up schedule Careful characterization of exposure (risk) and confounding factors Specific outcome reporting (assessments) with adjudication.

38. Registries - Epidemiology Embedded cohort study Identify eligible subjects Enroll subjects (consent) Establish a schedule of assessments to be conducted as part of routine care Send notices to patients when visits due Capture exposure (risk) and confounding factors Identify possible outcomes through medical reports, with subsequent adjudication.

39. Registries - Epidemiology A hybrid Establish an embedded cohort study. Also implement a formal prospective study in a sub-sample. The latter can serve as a quality check on the former.

40. Registries - Epidemiology LARGE Sample Size N needed to detect a rare outcome (e.g. fulminanthepatotoxicity, or angioedema) If risk is 1 in 10,000, need N = 29,956 to be 95% confident that at least one case will be observed. If wished to have 85% power to detect a 50% increased risk, at least 75 events required. N = 836,000 followed for 1 year!!

41. Conclusions Registry can provide superior descriptions of quality of care and distribution of factors in broad population of interest. Not as rigorous as a formal prospective epidemiologic study, but can form the basis for such studies. Affords opportunities for large sample sizes needed to detect rare outcomes.