580 likes | 902 Views
Paving an EBM Trail; From Journal Club to Publication. Corey Lyon D.O. Research Family Medicine Residency Kansas City, MO. Objectives. Exposure to a longitudinal EBM curriculum Obtain ideas on how to conduct a “Journal Club” that teach information mastery skills
E N D
Paving an EBM Trail; From Journal Club to Publication Corey Lyon D.O. Research Family Medicine Residency Kansas City, MO
Objectives • Exposure to a longitudinal EBM curriculum • Obtain ideas on how to conduct a “Journal Club” that teach information mastery skills • Discover ways to put these EBM skills into practice with publication opportunities, poster presentations, and Grand Rounds presentations • Introduction to FPIN membership • Introduction to a structured QI project workbook • Time for idea sharing
RFM Residency 13-13-13 community program Inner city, tertiary hospital 20 full/part time faculty
The Residency Review Committee Requires That: • All residents must participate in some scholarly activity • The program must provide supervision over this activity • Residents must have practical experience in data searching and grading, statistical methods, and application to practice • The training environment must be in compliance with EBM practices
How we learn • 3 ways we learn • 1. Construct new knowledge and understanding based on what they already know and believe. • 2. Learning tasks need to be structured and organized in some meaningful way to allow effective transfer of knowledge from one context to a new context. • 3. Learning requires multiple exposures to and complex interactions with knowledge.
Where do I want to end up…… • Identify objectives for an EBM curriculum • How to convert information needed into an answerable question • To become better consumers of the medical literature • Learn what to read • Learn critical appraisal skills • Synthesis the evidence – understand what it means • Grade the evidence • Know where to go and how to search for evidence • Put these EBM skills in motion before graduation
Now, how to get there…. • A longitudinal experience • Repeat exposure to the EBM concepts over 3 years • It is how we learn • Journal Club (change the name if you can) • critical appraisal of new research articles • Develop high success/high reward EBM projects • Publication opportunities • Presentation opportunities
Goals of Journal Club • Learn components of EBM • Ask answerable question • Literature search strategies • Determine relevance/validity of the research • Synthesizing the evidence – convert to user friendly statistics • Assigning a level of evidence grade • Determine how apply the evidence
Goals of Journal Club (cont) • Learn about new evidence that may change our practice • Learn about new evidence that influence clinical decisions • Provide a healthy, group learning environment • Continue to refine information mastery skills • Focus on learners needs, not publishers/pushers
Not Goals of Journal Club • How to do research (instead of how to use research) • How to do statistics (instead of how to interpret it) • Only finding flaws in articles • Teach all components of EBM in 1 session • Humiliate or bully learners by not knowing or using EBM skills
Steps in Journal Club • Convert need for information into an answerable question • Track down the best evidence to answer that question • Critically appraise for validity, impact, applicability • Integrate the critical appraisal with clinical experience and patient uniqueness • Evaluate
Components of every JC • PICO • Patient/population, Intervention, Comparison, Outcome • Relevance • Validity • LoE • Synthesis • How to apply? • Strength/weakness • Challenges (not bashing)
Structure of Journal Club • Consistent schedule • Monthly, bi-monthly • Develop process in finding an article • Consistent components • PICO, relevance, validity, LoE, applicability • Use worksheets as a tool • At least 1 teaching point with synthesis • ie Therapy – OR, NNT, CI, etc • Resident participation • Resident leader to help pick articles • Resident summarizes article
Journal Club Topics • Therapy • Diagnosis • Harm • Guidelines • USPSTF • EBM guidelines
More than just articles • Literature searching skills • POC tools • Clinical decision rules • Drug rep evaluation
Finding articles for JC • Always be on the look out • Use pre-synthesised resources • Daily Poems with Essential Evidence Plus • www.essentialevidenceplus.com • ACP Journal Club - www.acpjc.org • PURLs from JFP (FPIN) • AFP briefs • Dynamed- www.ebscohost.com/dynamed/weeklyUpdate.php • Journal Watch - www.jwatch.org • Evidence-Based Practice- www.ebponline.net • News media
EBM skills - PICO • Asking answerable questions • Transform it into PICO
PICO Question Exercise • You have a 52 y/o male in your clinic you diagnose with BPH, while discussing treatment options, he asks if Saw Palmetto is a good treatment option • P: Adult males with BPH • I: Saw Palmetto • C: Prescription medications (Flomax, Hytrin, Cardura) • O: Symptom relief, reduction in symptom score • Search terms
EBM Skills - Relevance • Determine Relevance • Can use a worksheet • What questions can you ask to determine relevance? • Is the outcome one that patients care about • Is the problem studied common to your practice • Is the intervention feasible • Will the outcome change/confirm what you are doing
EBM skills - Validity • Determining Validity • Worksheet as a tool • Review the study design • SR – appropriate search, heterogeneity? • RCT – randomization? Similar groups? • Cohort – confounders, similar groups? • Glossary of EBM terms; seeing these terms help assess that the researchers used valid scientific method • www.aafp.org/online/en/home/publications/journals/afp/ebmtoolkit/ebmglossary/afppoems.html
EBM skills - LoE • Grading the evidence • Types of studies • CEBM table
. . .Let the Evidence Pyramid Be Your Guide As you move up the pyramid, the amount of available literature decreases, but increases in relevance to the clinical setting. Meta-Analysis Systematic Reviews Randomized Control Trials Cohort Studies Case-Control Studies Case Series Expert Opinion
Oxford Centre for Evidence-based Medicine Levels of Evidence (May 2001) http://www.cebm.net/index.aspx?o=1025
EBM skills - Synthesis • Synthesis • Causes the most fear among faculty • Practice, practice, practice • Utilize resources to help you prepare • Learning is planned • Evidence-Based Medicine; How to Practice and Teach EBM by Straus
EBM Skills - Synthesis • Convert study results into user friendly statistics • Number needed to treat/harm • Confidence Intervals • Likelihood Ratios • Use worksheets, online tools • Create 2X2 table • http://ktclearinghouse.ca/cebm/practise/ca/calculators/statscalc
User-Friendly Stats 2X2 Table for Outcome of Interest - + Experimental Treatment Placebo/ Comparison • a. Control Event Rate: Cer = c / c + d • b. Experimental Event Rate: Eer = a / a + b • c. Absolute Risk Reduction: ARR = Cer – Eer • d. Relative Risk Reduction: RRR = ARR / Cer • e. Relative Risk Ratio = Eer / Cer • Number Needed to Treat: NNT = 1 / ARR • http://ktclearinghouse.ca/cebm/practise/ca/calculators/statscalc
Example: User-Friendly Stats PrimaryEnd Points (first major cardiovascular events) + - Experimental treatment Placebo/ Comparison Control Event Rate: Cer = c / c + d = 251/251 + 8650 = 0.028 Experimental Event Rate: Eer = a / a + b = 142/142 + 8759 = 0.016 Absolute Risk Reduction: ARR = Cer – Eer = 0.028 – 0.016 = 0.012 Relative Risk Reduction: RRR = ARR / Cer = 0.012/0.028 = 0.43 Number Needed to Treat: NNT = 1 / ARR = 1/0.012 = 83 http://www.cebm.net/index.aspx?o=1160
Example: User-Friendly Stats Non – Fatal Myocardial Infarction Experimental treatment Placebo/ Comparison Control Event Rate: Cer = c / c + d = 62/62 + 8839 = 0.00696 Experimental Event Rate: Eer = a / a + b = 22/22 + 8879 = 0.00247 Absolute Risk Reduction: ARR = Cer – Eer = 0.00696 – 0.00247 = 0.00449 Relative Risk Reduction: RRR = 0.00449/0.00696 = 0.65 Number Needed to Treat: NNT = 1 / 0.00449 = 223 http://www.cebm.net/index.aspx?o=1160
EBM Skills - Assessment • Assessing clinical importance • Discuss whether the treatment/diagnosis effective • Also discuss how effective • P value – likelihood the difference is due to chance • RR – Relative Risk Reduction – magnitude of difference, but not necessarily clinical importance • AR – absolute reduction – better measure of clinical importance • NNT – better understanding of the magnitude of results
Special Sessions – Literature search • PubMed searches as a group • Search terms • Limits • PubMed tools • My NCBI • Collections • Citation matcher • Clinical Query • Mesh terms • Tips for accessing full text
Special Sessions – Literature search • Other Medline tools with your institution • Ovid • EBSCO • Essential Evidence Plus • National Guideline Clearinghouse • Library access
Special Sessions – Literature search • See one – Do one • Look Up Conference • Divide into small groups – 3-4 residents • Present group with question – “Saw Palmeto for BPH” • Convert to PICO – easier to figure out search terms • Assign different EBM Source to each group • PubMed, EE+, Cochran, NGC, JFP, Google, etc • Find best 1-2 articles • As large group – discuss results, assign a LoE, determine answer (if able) • Discuss success/difficulties with different sites
Special Sessions – POC tools • PEPID demonstration • Diagnosis, treatment, calculators, guidelines, etc • Other Clinical Decision Tools • EE+ tools • iPhone apps – MedCalc, Qx Calculate, AHRQ ePSS • FRAX, Framingham CV Risk, etc
Putting Skills in Motion • Scholarly projects with goals in mind • Utilize EBM skills emphasized in JC • Forming an answerable question • Literature search • Assigning a LoE/Grade the evidence • Synthesize the article • Come up with an answer
Putting Skills in Motion • Structure, structure, structure • High yield, high success • Relatively quick, attainable projects • Dedicated time • Not just on top of all other requirements
Project Examples • RFMR utilizes FPIN membership • Family Physicians Inquiries Network • Provides an avenue and tools for publication opportunities • Assist faculty and residents in becoming contributors to the medical literature – utilizing EBM skills • Writing Projects • eMedRef – fast track point of care topic reviews (PEPID) • Help Desk Answers – Concise answers to clinical questions
Help Desk Answers • 500 word manuscript • Resident and faculty co-author • Faculty need to publish first – do one, then teach one. • Work with Local Editor and Editor-in-Chief • Peer reviewed at another FPIN program • Complete in 24 hours over 12 weeks • Published in Evidence-Based Practice
Other Projects – Local Academy • Missouri Academy of Family Physicians Magazine • Quarterly “Resident Grand Rounds” • Resident choose the topic • Case Study, QI project, Review of medical condition (Pediatric Sports Concussion)
Writing Structure • RFMR example • Complete during a 4 week rotation • Give ½ day/week of admin time • Outline the timeline • 1st week – Literature review complete • Meet with faculty to review search, decide which article to pull • 2nd week – Review articles • Meet with faculty, discuss article findings, determine which articles to use by relevance, LoE, etc • 3rd week – write outline to article – meet with faculty • 4th week – Complete 1st draft – give to faculty with articles
Writing Structure - continued • Faculty role • Guide through writing process • Work toward achieving EBM goals – not publication • Literature search, synthesis, assign LoE/SORT, etc • Keep on timeline, scheduled meetings • Once EBM goals achieved, complete the heavy lifting • Edit before turning into editor/peer review • Assist with addressing peer reviewer’s and editors comments • Can be overwhelming • Keep the enthusiasm going
Other Projects-Quality Improvement • Takes a little more time to complete project • Still needs structure • Workbook developed • Focus on key steps of a QI project – PDSA cycle • Emphasize EBM skills • Know that the change is an improvement • Literature review • Synthesis and grade evidence • Utilize EBM guidelines • Develop timeline, keep to it; sustainability • Success is dependant on faculty involvement • Sustain enthusiasm
Keep the momentum going • Turn the project into a poster presentation • State academy’s annual conference • AAFP Scientific Assembly • STFM • NCFMR • Types of posters • Clinical Inquiries • Case Studies • QI/PI projects
RFMR 3 year success • 15 HDAs published in Evidence-Based Practice • 4 other publication types • MAFP, case series, etc • 8 QI projects completed • 12 posters presented at MAFP ASA • Two 1st place • Two 2nd place • One 3rd place • 9 HDAs pending; 1 MAFP pending, 1 QI project pending
Skills carry over • R-3 Grand Round Presentation • EBM expectations • Cite original research • Which means – literature search, synthesis, explain results, etc • Assign a LoE to all cited articles • Give key recommendations at end of talk with SORT grade • Similar to AFP articles • Faculty mentor assigned • Check list and time line followed • Evaluation forms to give feedback on the EBM components of presentation