Considerations for Statistical Analysis in Observational Comparative Effectiveness Research

1. Considerations for Statistical Analysis in Observational Comparative Effectiveness Research Prepared for: Agency for Healthcare Research and Quality (AHRQ) www.ahrq.gov

2. Outline of Material This presentation will: • Describe the key variables of interest with regard to factors that determine appropriate statistical analysis • Propose descriptive analysis or graph according to treatment group • Propose the model that will be used for primary and secondary analytical objectives

3. Introduction • When observational data are used in comparative effectiveness research, careful and often complex analytic strategies are required to adjust for confounding. • Statistical considerations • Descriptive Statistics/Unadjusted Analyses • Traditional Multiple Regression • Model Selection • Model Assumptions • Propensity Scores/Disease Risk Scores • Instrumental Variables • Missing Data Considerations • Time-Varying Exposures/Covariates

4. Descriptive Statistics and Unadjusted Analysis • Descriptive statistics • Continuous variables: measures of range, dispersion, and central tendency • Categorical variables: frequency (n) and percentage • Data distributions: Kaplan-Meier plots • Unadjusted analysis • Conducted to identify covariates associated with the exposure and/or study outcome (e.g., t-test) • Can provide a broad picture of study subject characteristics

5. Adjusted Analysis • Traditional multivariable regression • Control for potential confounding variables in the estimation of treatment effects • Useful when there is a sufficient number of outcome events per covariate and exposure is not infrequent • If exposure is common and the outcome rare, consider propensity scores. • If exposure is infrequent, consider disease risk scores.

6. Model Selection

7. Propensity Scores • Propensity scores measure the probability of receiving treatment (or exposure) conditional on observed covariates. • Propensity scores are favorable in studies with a common exposure and rare or multiple outcomes. • They can be used in subclassification or stratification, matching, and weighting. • They include covariates that are true confounders or at least related to study outcome.

8. Disease Risk Scores • Disease risk scores (DRSs) measure the estimated probability or rate of outcome occurrence as a function of covariates. • Estimation approaches: • Fit regression model for entire cohort, adjusting for exposure • Fit regression model for unexposed/referent group • Compute fitted values assuming unexposed/referent group for all study subjects • DRSs are favorable in studies having a common outcome and rare exposure. • DRSs are useful for effect modification by disease risk.

9. Instrumental Variables • Instrumental variables are measures that are causally related to exposure but unrelated to outcome and study covariates. • They are useful for adjusting for potential unmeasured confounders. • However, it may be difficult to identify a high-quality instrument. • They can be used in conjunction with traditional multiple regression and propensity score matching.

10. Missing Data Considerations • Observational studies commonly have missing data. • Missingness can be characterized by using exploratory data analyses. • Complete-case analysis for subjects with no missing data: • Can reduce sample size, limiting efficiency • Can result in potential bias if missingness is differential between groups • Imputation for missing completely at random or missing at random

11. Time-Varying Exposures/Covariates • Time-dependent Cox regression models can account for time-varying exposures and covariates. • However, difficult issues arise when both treatment and confounding variables vary over time. • Inverse-probability-of-treatment weighting can be used to estimate a marginal structure model. • This approach is a generalization of propensity score weighting to the time-varying treatment context. • Intent-to-treat analysis can be conducted in which exposure status is assumed throughout followup when treatment adherence is low.

12. Conclusion • Observational comparative effectiveness studies are often strongly affected by confounding. • Thoughtful application of statistical approaches can adjust for confounding and improve causal inference. • An appropriate analytical technique is based on assumptions. • Consider the effect of missing data on analyses. • Sensitivity analyses can address residual confounding.

13. Summary Checklist (1 of 3)

14. Summary Checklist (2 of 3)

15. Summary Checklist (3 of 3)