Critically Appraising a Medical Journal Article

1. Critically Appraising a Medical Journal Article Bill Burton, Ph.D. Associate Director Office of Educational Resources AECOM

2. What Is Critical Appraisal? • quickly exclude papers that are of too poor a quality to inform practice • systematically evaluate those that “make the grade” in order to extract their salient points A technique for judging the methodological quality of a study. It allows you to …

3. Why Is Critical Appraisal a Necessary Skill for Physicians? • 20,000 biomedical journals, publishing 6 million articles a year • Most of this is of limited relevance and utility • All studies have flaws • You can’t always depend on a good review • As physicians, you need to become “critical yet comprehending” readers of the medical literature

4. Critical Appraisal Is Part of Evidence-Based Medicine FIND (the best evidence) APPRAISE (its quality) APPLY (the findings to your practice)

5. How to Become a “Critical” Reader • Don’t accept everything an author says at face value • Pay special attention to the Methods section • Continually ask yourself questions (and try to answer them) as you’re reading an article • Do the numbers add up? • Do terms like “random selection”, “double-blind”, and so on accurately reflect what was done?

6. Critical Appraisal Involves3 Stages • UNDERSTAND the article • What was the question? • What methods were used? • What did the authors find? • Determine its VALIDITY • Were the study methods sound? • Determine its UTILITY • How useful (i.e., generalizable) are the findings to your practice and patients?

7. Worth Reading? Believable? How Do You Decide if an Article Is…

8. Use Your Knowledge of the Following to Evaluate an Article • Study design • Epidemiology • Hypothesis testing • Statistics

9. Review of Study Design • Observational Studies • Case report/series • Cross-sectional • Case-control • Cohort • Experimental Studies

10. Other Terms/Concepts You Should Be Familiar With • Null hypothesis (vs. research hypothesis) • Sources of confounding and bias • Statistical power • P-value (vs. alpha level) • Confidence interval • Intention-to-treat analysis • Sensitivity & specificity

11. 1 - Understand the Article • What is the research question? • What study design was used? • Who are the subjects and how were they selected? • What did the intervention consist of? • What are the measures, and how were they obtained? • How were the data analyzed? • What are the findings and conclusions?

12. 2 - Determine the Article’s Validity (Internal Validity) • Is the research question focused and clearly stated? • Were the methods appropriate given the question? • Did the investigators obtain a representative sample of the population? Did they have a large enough sample? • Were the groups (e.g., intervention and control) similar at the start of the study? • Were exposures/outcomes accurately measured to minimize bias? • Have confounding factors been accounted for in the design and/or analysis? • Are the conclusions validly based on the data and analysis?

13. 3 - Determine the Article’s Utility (External Validity) • Is the question a clinically important one? • Are the subjects similar to the patients you see? • Were all important outcomes considered (e.g., potential harms as well as benefits)? • The difference between the treatment and control groups was found to be statistically significant, but is it clinically significant? • Could you apply the same treatment or intervention to your patients?

14. You Need to Choose an Article That Is … • An original report of a research study (do not choose a review article) • From a professional journal • On a topic relevant to ambulatory care, and ideally dealing with an urban population • An investigation of a specific hypothesis