Literature searching & critical appraisal

1. Literature searching &critical appraisal Chihaya Koriyama August 15, 2011 (Lecture 2)

2. Why is literature search important?Literature search helps you to: • Know what has been already done • Get up-to-date information • Develop your research hypothesis • Design your study • Study design • How to collect the information • Data analysis

3. Previous studies are your best teachers Spend ENOUGH TIME on literature search BEFORE you start designing your study

4. On-line information sources • PubMed http://www.ncbi.nlm.nih.gov/sites/entrez • embase http://www.embase.com/ • Free Medical Journals http://www.freemedicaljournals.com/ • Popline http://www.popline.org/ • HINARI http://www.who.int/hinari/en/

5. How to do a literature search • Web site of Robert Gordon University, UK http://www4.rgu.ac.uk/library/howto/page.cfm?pge=25989 • Stages of the search • Record keeping and referencing • Sources of information • Search techniques • Evaluating the results of your search • Useful online tutorials and web pages

6. Key words Number of hits

7. See all related articles

9. Start with Review

10. When not sure of the keyword

11. Critical appraisal of scientific articles

12. What is critical appraisal? Critical appraisal is a systematic approach to: - reading, - understanding, - interpreting, - identifying the limitations of and - deciding upon the usefulness of results of scientific papers.

13. What is Journal Club? Presentation of an article related to the presenter's research, followed by discussion with other members. Discussion should focus on what can be learned from the article, not on the criticism of the article. * It can be weekly, biweekly, monthly, etc.

14. Can you find this information in the paper? Check the publication year, Journal name, and author(s) • What is the research question? • What is the study type (design)? • Selection issues • What are the outcome factors and how are they measured? • What are the study factors and how are they measured?

15. Can you find this information (continued)? • What important potential confounders are considered? • What is the statistical method in the study? • Statistical results • What conclusions did the authors reach about the research question? • Study limitations? • How do you apply the findings into your daily clinical practice or research? An ideal abstract contains the underlined information at least.

16. 3. Selection issues (1) • Are there selection biases? • In a case-control study, suppose you want to compare exercise habits. controls cases Are they reasonably comparable?

17. 3. Selection issues (2) • In an intervention study, how were subjects recruited and assigned to groups? • In a cohort study, how many reached final follow-up? Randomly?

18. 4. What are the outcome factors and how are they measured? • Are all relevant outcomes assessed? • Is there measurement error? Consider the sensitivity & specificity of each detection method. 5. What are the study factors and How are they measured? • Is there measurement error?

19. 6. Confounder(s) • Suppose, you want to compare the proportion of cigarette smokers between stomach cases and controls. • You found more smokers in the GC cases. I found a significant association between smoking and gastric cancer risk!

20. 6. Confounder(s) • However, you found that there were more males in GC cases than controls. • Since a proportion of smokers in males usually higher than that in females, your finding might be distorted because of the different sex distribution. After adjusting the effect of sex distribution, there was no difference in the association between smoking and GC risk.

21. 7. What is the statistical method in the study? • Are the results dealt with adequately? • The effects of confounder(s) should be controlled. Lecture of statistical analysis will be given later

22. 8. Statistical results Please check • Is a P value given? • Are confidence intervals (CI) given? • If results negative and CI not given, is the power of the test given?

23. Interpretation of P value • When P value is less than significance level, which is usually 0.05, we often “reject the null hypothesis” Statistically significant! Conversely, when P value is greater than 0.05, we conclude that the result is not statistically significant.

24. What does “no significance” mean? • No sufficient evidence to doubt the validity of the null hypothesis • We cannot reject the null hypothesis (we concede that there is no difference) • The statistical test does not prove the null hypothesis.

25. Interpretation of confidence intervals • All estimates have their corresponding confidence intervals. • Suppose, you want to know the mean cholesterol level in a population but you cannot examine all subjects. • You randomly select a part of the subjects and calculate the mean value. • You can also calculate its confidence interval.

26. Set of 95% confidence intervals of 100 experiments These five out of 100 results do not include the true value. True mean value Your result may be one of these results but nobody can tell you “which one”.

27. Confidence intervals of odds ratio • Odds ratio : a measure of the effect, the strength of the association between a study factor and outcome. Odds in the exposed group = Odds in the un-exposed group Odds ratio =1 : there is no difference in the disease risk between the exposed and un-exposed groups. Odds ratio > 1: disease risk in the exposed group is higher than that in the un-exposed group. Odds ratio < 1: disease risk in the exposed group is lower than that in the un-exposed group. Disease risk

28. Crude OR=(38x72)/(57x71) = 0.68 When it is adjusted by confounding factors・・・・ Campos et al. (J Cancer Res Clin Oncol 2008) Since “odds ratio =1” means that there is no difference in the risk between this group (Q4) and referent group (Q1) If 95%CI contains “1”: Not statistically different If 95%CI does NOT contain “1”: Statistically different