Observational Studies in Comparative Effectiveness Reviews: A Interactive Quiz

1. When To Select Observational StudiesInteractive Quiz Prepared for: The Agency for Healthcare Research and Quality (AHRQ) Training Modules for Systematic Reviews Methods Guide www.ahrq.gov

2. Using Observational Studies inComparative Effectiveness Reviews When should observational studies be considered for inclusion in a comparative effectiveness review? • Observational studies should never be considered for inclusion. • Observational studies should always be considered for inclusion as the default strategy. • Observational studies should be included only when there is sufficient evidence from randomized controlled trials (RCTs) to confirm the validity of RCT findings.

3. Using Observational Studies inComparative Effectiveness Reviews Which of the following is the first question to consider when deciding whether or not to include observational studies in assessments of benefits? • Will trials provide valid and useful information to address key questions? • Are there gaps in the available observational evidence? • Are there gaps in trial evidence for the review questions under consideration?

4. Observational Studies and Applicabilityof the Comparative Effectiveness Review A comparative effectiveness review to compare percutaneous coronary intervention with coronary artery bypass graft for coronary disease identified 23 randomized controlled trials. Your Technical Expert Panel has raised concerns that these 23 trials enrolled patients with a relatively narrow spectrum of disease relative to those having the procedures in current practice. What should you do? • Change your key questions to focus on the types of patients for whom research is available. • Complete your review, including only the 23 trials that were identified. • Expand your review to include observational studies.

5. When Observational Studies Are Needed Once gaps in the trial evidence to assess benefits have been established and you have decided to include observational studies, which of the following should you do? • Include all suitable observational studies found to answer the initial key questions. • Exclude trial data that can be answered by the selected observational studies. • Refocus the study question on the gaps in trial evidence.

6. What Types of Observational Studies Are Best for Assessing Harms? In the absence of sufficient trial data, which types of observational studies should routinely be searched for and included in comparative effectiveness reviews as evidence of harms? • Case reports • Cross-sectional studies • Cohort and case-control studies

7. Assessing Bias in Observational Studies What is confounding by indication? • A type of detection bias • A type of attrition bias • A type of selection bias

8. Summary • Observational studies should always be considered for inclusion in comparative effectiveness reviews (CERs). • When considering whether to include observational studies in assessments of benefits, determine first if there are gaps in the trial evidence under review. • When assessing harms in comparative effectiveness reviews, investigators should routinely include cohort and case-control studies. • Once gaps in trial evidence to assess benefits have been established and the decision has been made to include observational studies, reviewers should refocus study questions on the gaps in trial evidence and look for observational studies that address the gaps. • Studies with a high risk of bias due to confounding by indication are usually not suitable for inclusion in CERs.

9. Authors • This quiz was prepared by Dan Jonas, M.D., M.P.H., and Karen Crotty, Ph.D., M.P.H., members of the Research Triangle Institute–University of North Carolina Evidence-based Practice Center. • The module is based on chapters 4 and 8 in version 1.0 of the Methods Guide for Comparative Effectiveness Reviews (available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/60/294/2009_0805_principles1.pdf).