1 / 42

Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea

Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea. Kelly Near Andrea Horne Karen Grandage Claude Moore Health Sciences Library. Overview. Proliferation of electronic resources Busy health professionals and students

chancellor-swanson
Download Presentation

Collecting User Preferences for Web-Based MEDLINE Training: A Capitol Idea

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Collecting User Preferences for Web-Based MEDLINE Training:A Capitol Idea Kelly Near Andrea Horne Karen Grandage Claude Moore Health Sciences Library Mid-Atlantic Chapter of the Medical Library Association 2002 Annual Meeting

  2. Overview • Proliferation of electronic resources • Busy health professionals and students • Need for continual end-user training and support, available any time & any place

  3. Tutorial Study Objectives: • Compare user preferences • Gain experience with new technologies for delivering online training • Evaluate the educational effectiveness of each tutorial format • Evaluate the cost/benefit ratio for each tutorial format

  4. Background • Advancements in technology • Web frames • Streaming audio & video • Flash • Java Script Applets

  5. Background – MEDLINE Training • Still our “bread and butter” resource • Medical and Nursing School curriculum training sessions • Monthly classes • Individual consultations

  6. Planning • Always allow more time than you think you’ll need! • We began with the assumption that online tutorials were an important part of educational delivery

  7. Planning • Tutorial formats were chosen based on - • learning style theory • our skill & comfort level with the necessary technology

  8. Tutorial A Tutorial A • Interactive • Created in Web frames • Search instructions in the left frame • “Live” version of the MEDLINE database active in the right frame

  9. Tutorial B Tutorial B • Short animated screen capture illustrations of commonly performed database tasks • Audio narrative

  10. Tutorial C Tutorial C • Classroom format • Streaming video and audio • Clips of an instructor interspersed with screen shots of the database

  11. Study Design • Wanted our study participants to be a cross-section of our usual MEDLINE class students • 10 participants were randomly assigned to each tutorial • We did not determine their MEDLINE searching abilities prior to the testing • None of the tutorials were meant to be as detailed as an actual class

  12. Debate: - should they answer specific search questions, such as “what is the symbol used to truncate in Ovid MEDLINE” or - should they conduct a complete search on a given topic, and have it measured against a “gold standard”?

  13. Search Question • Use MEDLINE 1996 – present Perform a comprehensive search on the question: Is there a connection between head injuries and depression in children?

  14. Six Steps to a Good Search • Rated each search on a 6 point scale • 1 point for selecting the MeSH term “craniocerebral trauma” • 1 point for exploding it • 1 point for selecting the MeSH term “depression” • 1 point for NOT exploding it! • 1 point for combining the terms with AND • 1 point for applying the age limit

  15. Explode ! Gold Standard

  16. Developing Tutorial A • Key features: • HTML frames • Ovid Jumpstarts • Products used: • HTML Assistant • Notepad • Cost = 0 • Challenges: • Condensing a full class down to key learning points and brief descriptions  Key:=development time$=money spent

  17. Developing Tutorial B • Key features: • Short video & audio clips with text • Products used: • SnagIt! • Microphone and headset • Cost = • $39.95 for single user copy $ Key:=development time$=money spent

  18. Take 5! Developing Tutorial B • Challenges: • Time required to write a concise script • Time for the multiple takes needed to do a clean audio track … • especially using non-union help!

  19. Starting a Subject Search This example will demonstrate a search for articles about oral cancers in teenage boys who use chewing tobacco. Begin by separating your search topic into individual concepts.Type your first search concept into the box labeled Enter Keyword or Phrase. For our search, we begin by typing in the keywords "oral cancer". Be sure that the checkmark is in the box by Map Term to Subject Heading Next TopicReturn to Tutorial Home

  20. Developing Tutorial C • Key features: • Used our classroom as the “stage” • Hired a “professional actress” so fewer retakes were necessary • Products used: • Adobe Premiere • Cost = $399 $$$$ Key:=development time$=money spent

  21. Developing Tutorial C • Challenges: • iMovie and Final Cut Pro didn’t allow us to intersperse SnagIt video clips with instructor video • Next tried videotaping the screen in the classroom as the MEDLINE search was being done • Adobe Premiere allowed us to combine the video and the SnagIt clips into one product

  22. Recruitment • Sent email to medical students in all 4 years inviting them to participate in the study • Posted recruitment flyers around the Library, Medical School and neighboring Research Buildings • Word-of-mouth seemed to help • So did the FREE lunch!

  23. Pre-Testing • Pre-tested the process with 3 library staff • Identified problems with inconsistent instructions • Refined test questions and tutorials based on this input

  24. Testing • Randomly assigned 10 participants to each tutorial • The tutorials were taken in a library classroom • After completing the tutorial, they were each given the same search to complete in Ovid MEDLINE

  25. Interviews • Participants were asked12 questions • Examples • Have you ever taken a MEDLINE class before? • Did you have any technical problems with the tutorial? • Did you refer back to the tutorial to complete the steps for the test search?

  26. Free Lunch • Testing over the middle of the day worked very well as the lure of a FREE LUNCH was irresistible to all!

  27. Test Scores Number of questions Number correct

  28. ANOVA Test Results P value needs to be below 5% - this is 45%

  29. Explode Number of Participants Number Exploding

  30. Chi Square Analysis • Chi Square analysis included 3 separate tests • Only 1 showed a significant difference • No significant statistical differences were found overall

  31. Tutorial Preferences • Overall, interview results showed positive acceptance for all three tutorial types A B C

  32. Reported Learning Preferences Class Online Tutorial 4 22 3 Printed

  33. Lessons Learned … • Only 8 users had not used MEDLINE before • We looked at this data to determine if there was a difference in the scores between new and experienced MEDLINE users • Did not find any difference between the 2 groups

  34. Lessons Learned … • May need a larger sample size • AVI files (from SnagIt!) are too large • Try Snapz Pro (Macintosh only) for video captures

  35. One Caveat … • Ten days before we were to start testing - we found we needed approval from the University’s Human Investigation!

  36. Conclusions • Scores = no difference • Next, we looked at production costs and time and found … MAJOR DIFFERENCES

  37. …and the Winner is -- Tutorial A!

  38. Future Directions

  39. Tutorial A = the one to trust!

  40. Collecting User Preferences for • Web-Based MEDLINE Training:A Capitol Idea • Kelly Near, Andrea Horne, Karen Grandage Questions ?

More Related