EVIDENCE BASED PRACTICE AND ITS IMPACT ON THE BOC EXAMINATION

1. EVIDENCE BASED PRACTICE AND ITS IMPACT ON THE BOC EXAMINATION Athletic Training Educators’ Conference Washington D.C. February 25, 2011 Susan McGowen, PhD, EMT, ATC University of New Mexico Bernadette Olson, EdD, ATC South Dakota State University

2. DRIVING CHANGE IN ATHLETIC TRAINING PRACTICE: Implementing Evidence Based Practice Skills into Athletic Training Education Curriculum • Educate future practitioners, students should successfully demonstrate: • Use of evidence based practice during didactic/clinical experiences. • Implement EBP strategies into their own entry level practice careers. • Stretch: • As new practitioners, students will participate/drive change in the organizations for which they work to advance EBP environments. • Educate current practitioners • Educate the educators: • Are we ready to serve as role models?

3. And the survey says… • 72% of participants completed the survey… • Program Directors (51%) • Faculty (41%) • Clinical Instructors (8%) • Need participation from all in curriculum to integrate EBP

7. Why We Should Care - Improving Patient Outcomes • (1) It is ethical: Care about patients/do no harm • (2) Smarter patient(s): We are in an information era – • patient advocacy • (3) It is legal: We are held to a standard of care... • (4) It’s contemporary: Everyone else is doing it… • “The train is leaving the station…” • (5) Profession Integrity: Making our mark as experts… • (6) It is a competitive market: If you don’t get participants back faster, then someone else will. • (7) Cost Effectiveness: We can’t afford to continue treating conditions the same way. • (8) How the public views continuing competence: From seat time to demonstrating a change in patient outcomes.

8. Barriers to EBP (in the classroom & practice) • Knowledge base of educators and practitioners… • Do we know enough to be teaching practices? • Where do we get our information from? • How much EBP is out there?... • Is the research available for us to answer our clinical questions? • Relationship to theoretical research… • Time to… • learn • practice • disseminate

9. Evidence-based practice is the integration of best research evidence with clinical expertise and patient values to improve patient outcomes. Best Research Evidence √ Clinically relevant research - patient-centered clinical research √ Diagnosis, Prognosis, Therapy, Rehabilitation, Prevention. Clinical Decision Making Patient Values/Patient Circumstances √ Unique preferences, concerns and expectations that each patient brings to a clinical encounter and that must be integrated into clinical decisions if they are to serve the patient. Clinical Expertise √ Clinical Skills/Past Experience √ Rapidly identify each patient’s unique health state and diagnosis, individual risks and benefits of potential interventions, and personal values and expectations. Improved Patient Outcomes Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001)

10. Best Research Evidence Critical Appraisal… Clinical Expertise Patient Values Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001)

11. What are we preparing our students to do? Be Consumers… Be Producers… • Inquisitive practitioners/lifelong learners • Effectively locate and analyze literature • Making decisions • Collaborating with other practitioners • Communication networks, Site practice guidelines, Practice patterns, Journal Clubs • Is it time to participate in a study design? • Randomized control trials • Cohort Studies • Case Studies • Disseminate information to benefit other providers • Via publications • Via presentations • Teaching others how to participate in EBP practice

12. At minimum, an entry-level clinician should be able to… • Formulate sound clinical questions • Locate literature relevant to question • Critically appraise literature: • A single article/study or arranged around a topic • Quantitative or Qualitative • Reflect and determine if this affects practice • Others…

13. Take (10) • How are you starting the conversations in your programs…

14. What conversations is the BOC having? How does EBP influence the “business” of the BOC? References Role Delineation Study / Practice Analysis Entry-Level Examination Remember – The BOC examination does not drive change, education drives change Evidence used in the BOC examination reflects what is currently accepted as common practice in athletic training.

15. BOC References • How is the BOC Examination Reference List determined?

16. Program Director Reference Survey • January 2011 • Survey Purpose: • To determine references most commonly used by entry-level ATEPs • To determine preferred references when writing, reviewing and validating items within exam development process by BOC Exam Development Committee (EDC) • Overall: program director references coincided with what EDC members preferred

17. Program Director Reference Survey Summary

20. Relationship between EBP and RDS

21. Role Delineation/Practice Analysis • The RD/Practice Analysis is: • A review of existing RD/Practice Analysis material • Subject matter expert knowledge • A large scale survey • Findings are used to evaluate and support an entry level athletic trainer competency test

22. RD/Practice Analysis 6 • Example from Domain 3: Immediate and Emergency Care

23. How EBP affects the Exam Process

24. What does an evidence based practice environment look like NOW? Reflected in current 6th Role Delineation Study (RDS) BOC Examination

25. Design of the BOC Examination…

26. Multiple Choice Item Development • Triangulation Subject Matter Experts Statistical Analysis Double Reference

27. Multiple Choice Item Development • Subject matter experts • Represent 5 domains • Items based on: • Importance • Criticality • Frequency

28. Multiple Choice Item Development • Importance • Is competence in a domain essential for a minimally competent certified athletic trainer? • Criticality • Can adverse effects result if the certified athletic trainer is not competent in a domain? • Frequency • What percentage of time does the certified athletic trainer spend performing duties associated with the performance domain?

29. Breakout Session • Divide into groups • Construct 3 multiple choice items • Recall • Analysis • Application • Include question, correct answer and four distracters (incorrect responses)

30. Multiple Choice Item Principles • Entry-level • Clear and concise • Written in question form • Third person • Stem (question) stated in a positive matter • Distracters clearly wrong, but plausible • Double reference • Be aware of state/local/regional differences Adapted from the BOC Style Guide

31. MC Item Examples: Recall INITIAL ITEM REVISED ITEM • According to Richard Ray, there are specific budget types used by athletic training facilities to project future incomes and track expenses. Which of the following budget type requires an athletic trainer to list anticipated expenditures for specific categories of program functions, including equipment repair, insurance, and expendable supplies? Choose only one. • Fixed • Line-item • Lump-sum • Performance • Zero-based • What budget type requires an athletic trainer to list anticipated expenditures for specific categories of program functions? Choose only one. • Fixed • Line-item • Lump-sum • Performance • Zero-based (clear and concise stem)

32. MC Item Examples: Analysis INITIAL ITEM REVISED ITEM Section C of the budget shown below includes which type of expenditures? Choose only one. Capital Line item Departmental Cash Operating (use of abbreviations and trademarked terms)

33. MC Item Examples: Application INITIAL ITEM REVISED ITEM • A patient presents to the athletic trainer with pain, moderate effusion, and instability of their left knee after experiencing an incident where they felt a “pop” while making a quick lateral movement about a planted foot. Which of the following structures are MOST likely affected? Choose only one. • ACL • MCL • LCL • ATF • CF • A patient presents to the athletic trainer with pain, moderate effusion, and instability of their left knee after experiencing an incident where they felt a “pop” while making a quick lateral movement about a planted foot. Which of the following structures are MOST likely affected? Choose all that apply. • Anterior cruciate ligament • Medial collateral ligament • Lateral collateral ligament • Meniscus • Patella Tendon (length of distracters, wrong but plausible, abbreviations)

34. *Influence of Vision for the Profession (what do we want to look like in 5 yrs / 10 yrs) *Influence of Educators – Drive Change in Practice and Culture *Influence of Current Practitioners What does an evidence based practice environment look like in 5 years? Influenced by Professional Development we seek… Influenced Changes in Professional Education (4th Competencies and Proficiencies) Reflected in 7th (RDS)/ Practice Analysis

35. Closing Comments… The BOC does not create evidence The examination reflects what is currently accepted as common practice in athletic training

36. Discussion THANK YOU!