1 / 27

Assessing the efficacy of EBM teaching in a clinical setting

Assessing the efficacy of EBM teaching in a clinical setting. Gardois P., Grillo G., Lingua C., Jourdan A., Fronteddu S., Piga A. Turin – Italy. Background: evaluating IR and EBM skills. Growing popularity of EBM practice  Growing number of EBM courses

rafi
Download Presentation

Assessing the efficacy of EBM teaching in a clinical setting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing the efficacy of EBM teaching in a clinical setting Gardois P., Grillo G., Lingua C., Jourdan A., Fronteddu S., Piga A. Turin – Italy Santander, 9. EAHIL Conference

  2. Background: evaluating IR and EBM skills • Growing popularity of EBM practice •  Growing number of EBM courses • Many methods for teaching and learning evaluation • But... only few existing data on the application of the EBM skills in the clinical practice, mainly when: • the working environment is not EBM-oriented • there is an interest in long-term results Santander, 9. EAHIL Conference

  3. Objectives • To assess the long term effect of an educational program for health professionals, based mainly on information retrieval and EBM principles • Secondary goals: • to understand the role of EBM methods and instruments in clinical practice • to assess users’ perception of the main barriers to EBM practice Santander, 9. EAHIL Conference

  4. Our educational program… • Institution: Pediatrics and Gynecology Departments of Turin University (Italy) • Number of courses: 6 (5 with CME credits). • 4 focused on medical information retrieval: • theory of databases • basics of information retrieval • basic and advanced use of medical databases and search engines • bibliography formatting software • 2 specifically EBM oriented: • advanced IR skills • basic statistics • study design • critical appraisal • Time span: 2001-03 to 2003-06 Santander, 9. EAHIL Conference

  5. …Our educational program • Average duration: 16.5 hours (min 11, max 28) • 7 teachers and 6 tutors (librarians, clinicians, statisticians, epidemiologists) • 126 unique users (140 total); categories: • MDs, postgraduating doctors • Nurses, midwives • PhDs (chemistry, biology, …) • Other technical staff • average learners/facilitators rate: 3.9 (min 2.7 max 5.3) Santander, 9. EAHIL Conference

  6. Methods: 1. IR skills durability assessment • The multiple choice questionnaire administered after the course was administered again in may 2004 (11 to 38 months after) •  assessment of the Delta (Δ) between the two results Santander, 9. EAHIL Conference

  7. Methods: 2. EBM skills durability assessment • The first part of the Fresno Test (validated in 2002; see Ramos – BMJ 2003) was administered to test EBM skills in 4 areas: • Clinical question • Selection of bibliographic sources • Selection of study design • MEDLINE search strategy Santander, 9. EAHIL Conference

  8. Methods: 3. EBM practice assessment • Qualitative questionnaire with the following main categories of items: • Ideal and real percentage of clinical activity devoted to EBM • Relative frequency of use of different bibliographic sources • Experience of barriers and facilitators for EBM practice Santander, 9. EAHIL Conference

  9. Methods: users invitation and questionnaires completion • Users approached by letter and – after 1 week – by e-mail, explaining the study purpose and methods • Up to 5 phone calls during 3 weeks to make an appointment for questionnaire completion • Questionnaires administered individually in the library, without the use of computers. • 45 minutes for completion Santander, 9. EAHIL Conference

  10. Methods: statistical analysis • Software used: StatSoft Statistica 6.1 for MS Windows Santander, 9. EAHIL Conference

  11. Results • 1 Basic statistics • 1.1 Response rate • 1.2 Demographics (gender, age, role) • 2. Descriptive statistics • 2.1 Δ first – second multiple choice questionnaire • 2.2 Fresno test results • 2.3 Average responses for any variable in the EBM practice questionnaire • 3 Correlations between: • 1.2 + 2.3 vs 2.1 + 2.2 Santander, 9. EAHIL Conference

  12. Basic statistics: response rate • 70 / 126 users  55,6% response rate • N = 70 • Based on reported opinions the questionnaire was considered “difficult”. • A higher response rate might have been reached by accepting “remotely completed questionnaires”, but the data reliability would have been lower Santander, 9. EAHIL Conference

  13. Demographic data • Gender: F 71%, M 29% • Age: Mean = 41,03, StdDv = 8,70, Max = 62, Min = 27 • Role: 72% doctors, 28% other roles Santander, 9. EAHIL Conference

  14. Delta between 1st and 2nd completion of multiple choice learning evaluation questionnaire: • delta%: N = 70; Mean = -19,43; StdDv = 19,02 Santander, 9. EAHIL Conference

  15. Barriers • The main barriers affecting the results of the 2nd completion of the multiple choice questionnaire are: • Finding contradictory results in the literature • Knowledge of English • Insufficient number of PCs Santander, 9. EAHIL Conference

  16. Barriers • The main barriers affecting the Δ pct between 1st and 2nd completion of the multiple choice questionnaire is insufficient access to PCs Santander, 9. EAHIL Conference

  17. Results: Fresno test Theaveragetotalscorewas54,99 (valid N 46, Std.Dev. 24,24). Santander, 9. EAHIL Conference

  18. Barriers • The main barriers affecting the Fresno test total score are: • Lack of institutional support • Lack of statistics skills • Insufficient search strategy building abilities • Poor knowledge of English language Santander, 9. EAHIL Conference

  19. Results: EBM practice ideal and real With a valid N of 66, the average pct of time that our professional would devote to EBM practice was 23,06 (20,10 Std.dev), the real pct of time they devote is 6,29 (7,56 std.dev). Santander, 9. EAHIL Conference

  20. EBM practice: ideal and real • The real and ideal practice of EBM are correlated. Users who consider more important EBM practice tend to have a higher percentage of real EBM practice Santander, 9. EAHIL Conference

  21. Comment • Time is an important barrier as to practice EBM skills. There is a significant correlation between the two variables Santander, 9. EAHIL Conference

  22. Main bibliographic sources searched(pct. total weekly research hours, valid N = 67) • MEDLINE31,00 (25,78 std.dev)I • Proceedings16,40 (18,22 std.dev)I • Journals16,30 (13,38 std.dev)I • Colleagues10,07 (16,62 std.dev)I Santander, 9. EAHIL Conference

  23. Main barriers to EB practiceAVG points on 1-5 Likert scale:“totally disagree” to “totally agree” • Time3,4 (1,2 Std.Dev.) • Isolation3,2 (1,1 Std.Dev.) • Lack of institutional support 3,2 (1,3 Std.Dev.) Santander, 9. EAHIL Conference

  24. Main facilitators for EB practiceAVG points on 1-5 Likert scale:“totally disagree” to “totally agree” • Time4,0(0,8 Std.Dev.) • Critical appraisal skills4,0(0,9 Std.Dev.) Santander, 9. EAHIL Conference

  25. Other remarks • Users who had good 1st questionnaire results had a greater probability to have also good 2nd questionnaire results • Users who had good 2nd questionnaire results had a greater probability to have also good Fresno test total score results •  Good internal coherence of the assessment methods of the study Santander, 9. EAHIL Conference

  26. Conclusions • EBM skills tend to be lost after some time • Our users tend to consider ideal EBM practice very important, but the real practice of EBM is significantly lower than the ideal one • The main barriers to EBM practice, and the more related to the results of our tests are: time, access to PCs, isolation and lack of institutional support Santander, 9. EAHIL Conference

  27. Further research • We need larger, multicenter studies to reach more reliable and relevant results • We need to test if – for the practicing professional – regular recalls of EBM concepts are useful • We need to study more extensively the influence of barriers vs EBM practice and to take consequent action Santander, 9. EAHIL Conference

More Related