Introduction to Best Evidence Medical Education

1. Introduction toBest Evidence Medical Education

2. The Truth Is Rarely Pure And Never Simple Oscar wild 1895

3. Systematic Review OR synthetic research • Is main procedure for synthesis of clinical evidence

4. A ‘review’ is the generic term for any attempt to synthesis the results and conclusions of two or more publications on a given topic.

6. Systematic Review: What is it? • A summary of several research articles on one topic presented in table and narrative formats • Qualitative rather than quantitative analysis • Subjective interpretation • Time consuming process • Strongest form of evidence

7. What are the advantages of a systematic review? • Condensed: allowing the reader to access consolidated results of huge volumes of information; • Objective: reducing (though not eliminating) the risk of bias and error • Balanced: including a broad range of studies which are identified via a thorough and systematic search strategy; • Verifiable: incorporating transparent processes that allow the reader to know exactly how the conclusions were reached; • Replicable: using a structured methodology. • Flexible: can be updated on a regular basis • Dynamic: in identifying areas that are under-researched or in identifying new • research questions; • Readable: presented in a format that is easy to read and understand.

8. Rationale for systematic reviews • We need reliable information • There is too much information around • We need reviews of existing information • Reviews can be unscientific and biased in the way they collect, appraise and summarize information

9. A pilot systematic review and meta-analysis on the effectiveness of Problem Based Learning

10. A systematic review of computer-assisted learning in endodontics education

11. Increasing Knowledge

12. Not all of this information is valid or useful for patient care A little more than 1% is both rigorous and clinically relevant.

13. Most research published in medical journals is too poorly done or insufficiently relevant to be clinically useful

14. Too much information, too little time • There is simply too much information around for people to keep up to date. • On top of this, high quality information is often not easy to find.

15. Review articles

16. Some reviews are usually based on narrative or commentary and are produced by a ‘content expert’

17. Professor Paul Knipschild has described how Nobel prize winning biochemist Linus Pauling used selective quotes from the medical literature to "prove" his theory that vitamin C helps you live longer and feel better.

18. When Knipschild and his colleagues searched the literature systematically for evidence They found that

19. One or two trials did strongly suggest that vitamin C could prevent the onset of the common cold • There were far more studies which did not show any beneficial effect.

20. Unfortunately, expert reviewers often : • Make conflicting recommendations • Their advice frequently lags behind • Inconsistent with the best available evidence.

21. Minimizing bias We need to do as much as possible to minimize the effects of anything that will cause the results to deviate from the truth.

22. GOALS • Briefly Describe Synthetic Research • Systematic Review Teaching Package • Meta-analyses

23. Medical Education UME GME CME UME knowledge-focused GME apprenticeship, better differentiated CME discrete educational interventions Context, Content, Method, Setting Improved Learning Task-specific Competencies, Meta-competencies Essential Personal Characteristics Knowledge, Skills, Attitudes Improved Practice Safe, Timely, Patient-centered Effective, Efficient, Equitable Improved Outcomes Level of Health, Distribution of Health Responsiveness, Fair Finance

24. Opinion Based Teaching Evidence Based Teaching The concept of best evidence medical education • A more helpful view of evidence-based teaching is of it as a continuum between 100% opinion-based education at one end of the spectrum where no useful evidence is available, and 100% evidence-based education at the other where decisions can be taken on the basis of detailed evidence.

25. Hart (1999) has suggested that Taking a best evidence-based approach to medical education questions forces educators to: • comprehensively and critically appraise the literature that already exists in the area, and categorize the power of the evidence available • identify the gaps and flaws in the existing literature and suggest (and if possible carry out) appropriately planned studies to optimize the evidence necessary to make the proposed educational intervention truly evidence based.

27. What is the methodology behind a systematic review? • 1. Formulating the question • Should address the Patient group, the Intervention, the Comparison intervention, and the Outcome – PICO

28. PICO exa: • Among children under the age of five living in rural settings (patient), does the use of insecticide treated bed nets (intervention) lead to lower malaria prevalence rates (outcome) as compared to when nontreated nets are used (comparison intervention)?

29. www.cochrane.org/resources/handbook • www.cochrane.org/resources/openlearning

30. What is the methodology behind a systematic review? • 2. Search and inclusion of primary studies • Restricting a search to a general database(such as MEDLINE) is clearly insufficient; a review group found that this tends to identify only about half of all randomized-controlled trials (RCTs).

31. Where to locate primary studies? • Major databases • Reference of studies identified in databases • Unpublished studies (in order to avoid publication bias) • Interviews with the authors of original studies when data is missing • Non-English language studies

32. What is the methodology behind a systematic review? • 3. Quality assessment and data extraction • studies are appraised for their methodological error, particularly to identify any possible sources of bias • usually involves two independent reviewers

33. What is Critical Appraisal? • The process starts with careful examination of all aspects of the studies selected for inclusion in the systematic review. • This breaks down the components of the study to evaluate characteristics of participants, outcome measures used, completeness of study follow up, and appropriateness of statistical measures. • Critical appraisal – which lies at the very heart of a systematic review – requires dedicated time and expertise

35. What is the methodology behind a systematic review? • 4. Synthesis and summary of study results • If applicable, this is the stage where a meta-analysis may be used to pool statistical results from the various studies under review.

36. What is the methodology behind a systematic review? • 4. Synthesis and summary of study results • The aim is to combine the multiple findings in order to reach a conclusion on the clinical effectiveness of the intervention under consideration..

37. What is the methodology behind a systematic review? • 4. Synthesis and summary of study results • When a difference of effect is detected across various studies, the source of this divergence can then be analyzed

38. What is the methodology behind a systematic review? • 4. Synthesis and summary of study results • “limitations of the primary studies … may include issues relating to design flaws. Limitations of the review itself may include issues such as inclusion of only English language studies or inability to accurately interpret the summary estimates due to heterogeneity.”

39. What is the methodology behind a systematic review? • 5. Interpretation • synthesized study results are interpreted and the limitations of both the review and the studies within it are discussed

40. What is the methodology behind a systematic review? • 5. Interpretation • This allows the reader to judge personally the value of the review and its conclusions. • Finally, the review discusses the practical implications and applicability of the findings.

41. Where can I find systematic reviews? • The Cochrane Collaboration Reviews are available in two formats: on-line (www.cochranelibrary.com) or on CD-ROM

42. Writing the review • Guidelines on how to write reviews & meta-analyses: • QUORUM statement* • For meta-analysis of RCTs • MOOSE guidelines** • For meta-analysis of observational studies

44. The complete list of groups, which are hosted in centers around the world, is available at www.cochrane.org/contacts/entities.htm#centres.

45. Where can I find systematic reviews? • 2-The Campbell Collaboration • its focus on the healthcare sector http://www.campbellcollaboration.org

46. Campbell Collaboration consists of six Coordinating Groups, each with an area of specialization. • The Campbell Collaboration Crime and Justice; • The Campbell Collaboration Education; • The Campbell Collaboration Social Welfare; • The Campbell Collaboration Methods; • The Campbell Collaboration Communication and Internationalization • The Campbell Collaboration Users Group.

47. A systematic review of systematic reviews of homeopathy

48. SYNTHESIS AND SUMMARY OF STUDY RESULTS(META-ANALYSIS)

49. Pooling and pooled estimate • Forest plots display effect estimates from each study with their CI, and provide a visual summary of the data

50. Statistical models for pooling • random effects model • fixed effects model