1 / 72

Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project

Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project. James J. Cimino, M.D. Department of (Bio)Medical Informatics Weekly Seminar March 6, 2003. Studying Clinician Information.

barny
Download Presentation

Anticipating and Satisfying Clinician Information Needs: The Infobutton Manager Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Anticipating and Satisfying Clinician Information Needs:The Infobutton Manager Project James J. Cimino, M.D. Department of (Bio)Medical Informatics Weekly Seminar March 6, 2003

  2. Studying Clinician Information • Covell DG, Uman GC, Manning PR. Information needs in office practice: are they being met? Ann Intern Med 1985 Oct;103(4):596-9 • Other studies: Osheroff, Forsythe, Gorman, Ely… • Types of needs • Patient-specific • General knowledge • Institutional • Resources • Medical record • Colleagues • Journals • Books

  3. Do Clinicians Satisfy their Needs? • Needs are underestimated • Information needs may not be recognized • Resources are outdated or unavailable • Computers not used • Resolution may be deferred • Unsatisfied information needs are a potential source for errors

  4. Needs Related to CIS Use? • Very small part of the clinician’s day • Very important part of the day • New information stimulates needs • Decision-making moments • Opportunity to resolve needs (using a computer) • Context might predict needs • Context might help resolve needs • Needs in this setting have not been studied

  5. Context-Dependent Information Needs ! ? Age Sex Role Training Task Data Institution Context

  6. Hypothesis: • Information needs are stereotypical • Information needs can be predicted based on context • Appropriate resource can be identified (automatically or in advance) • Resource can be queried automatically • Information from context can be used for query • “Infobuttons”

  7. A Brief History of Infobuttons • 1991: “Medline Button” translated ICD9 to MeSH and performed Medline searches • 1994: World Wide Web • 1994: First CPMC Web-based CIS • 1996: WebCIS • 1996: Infobuttons link microbiology and medication data to PubMed and Micromedex • 1997: Infobuttons for patients (PatCIS) • 2002: Handheld, wireless infobuttons (PalmCIS)

  8. Infobutton Manager Project • Study information needs of CIS users • Characterize needs based on context • Develop way to anticipate needs • Develop ways to resolve needs • Deploy anticipation/solution • Evaluate

  9. Infobutton Manager Team Members • Jim Cimino • Jianhua Li • Sue Bakken • Vimla Patel • Mark Graham • Leanne Currie • Mureen Allen • Anne-Marie Ramierez • Your name here?

  10. Methods • Determine information needs • WebCIS log files • WebCIS user questionnaires • Observational studies • Characterize information needs • Build Infobutton Manager (IM) • Build infobuttons • Populate IM database • Integrate IM with WebCIS • Evaluation

  11. Insert WebCIS Demo Here

  12. WebCIS Log File Methods • Health Resources page in WebCIS • Infobuttons in WebCIS • Problems: • Which user selected which resource? • What was the user was doing? • Solution: look at previous activity at same IP address

  13. WebCIS Log File Records Mar 2 16:16:46 webcis3-i syslog: |WebCIS|ciminoj|156.111.145.61|3131313 |lab^2003-03-02-05.19.00.819650|view Mar 2 16:18:55 webcis3-i syslog: |WebCIS|ciminoj|156.111.145.61|3131313 |healthresource|view Mar 2 16:18:58 webcis3-i syslog: |Resources|(nobody)|156.111.145.61|| |url^http://salk.cpmc.columbia.edu:70/mdxdocs

  14. WebCIS Log Files Results • Six months of log files • 2,607 users (1,034 per month) used a resource • 2931 IPs (2008 “156.x.x.x”) • User types: Housestaff: 51% Attendings: 34% Nurses: 5% Other: 10%

  15. WebCIS Log Files Results 38,763 uses of resources 14,036 outside WebCIS (2339/month) 24,727 within WebCIS (4,104/month; 4/user) 19,913 uses of 24 resources (3,319/month) 4,814 uses of 3 infobuttons (802/month)

  16. WebCIS Log Files Results Popular resources: Inside Outside Micromedex: 56.5% 51.7% Ovid: 7.4% 8.2% NYP Formulary 7.2% 7.1% Harrison’s 5.1% 5.1% PubMed 5.1% 5.1% Health Journals 5.0% 8.2% Medline Plus 3.3% 3.6% ICD9-CM 3.3% 2.9% Infobuttons Pharmacy/Micromedex 92.9% - Microbiology/PubMed 6.1% - Pharmacy/PubMed 1.0% -

  17. WebCIS Log Files Results Health Resources: 19,913 times in 105 contexts: Lab results: 52.1% Radiology results: 9.0% Clinical notes: 7.4% Pathology results: 2.8% Visit list: 2.1% Pharmacy: 2.0% Discharge summary: 1.9% 80.4% Infobuttons: 4,814 times in 2 contexts: Pharmacy: 93.9% Lab results (Micro): 6.1% 100%

  18. WebCIS Log Files Results Context: Lab Rad Note Card Path Vis Phar DSum MDX 53% 9% 3% 3% 3% 2% 2% 2% Ovid 49% 9% 6% 2% 3% 1% 1% 1% Formul. 52% 8% 6% 3% 3% 1% 3% 2% Harrison 49% 10% 5% 3% 4% 1% 1% 3% PubMed 48% 10% 7% 3% 5% 1% 3% 1% Journals 43% 8% 7% 6% 5% 2% 2% 1% Med+ 50% 8% 6% 3% 2% 4% 2% 1% ICD9 37% 12% 10% 2% 3% 2% 1% 2% Infobuttons MDX - - - - - - 100% - PubMed 86% - - - - - 14% -

  19. WebCIS Log Files Results Context: Lab Rad Note Card Path Vis Phar DSum MDX 56% 57% 61% 59% 49% 42% 9% 62% Ovid 7% 7% 8% 5% 8% 4% 1% 6% Formulary 7% 6% 6% 6% 7% 3% 1% 7% Harrisons 5% 6% 4% 5% 6% 3% 1% 7% PubMed 5% 5% 5% 4% 8% 2% 1% 3% Journals 4% 4% 5% 9% 8% 3% 1% 3% Med+ 3% 3% 2% 3% 3% 32% 1% 2% ICD9 1% 4% 4% 2% 3% 2% .2% 3% Infobuttons MDX - - - - - - 84% - PubMed 3% - - - - - 2% -

  20. WebCIS Log Files Results User: Housestaff Attending Nurse Other MDX 62% 61% 42% 27% Ovid 6% 7% >1% 1% Formulary 6% 5% 3% 5% Harrison 4% 3% 4% 3% PubMed 4% 5% 1% 1% Journals 3% 0% 3% 1% Med+ 2% 2% 3% 2% ICD9 3% 5% 1% 1% Statref 1% 1% 1% 3% PIER 0% 2% 3% 1% Infobuttons MDX 5% 6% 42% 53% PubMed 2% 1% 2% 1%

  21. WebCIS Usage Logs Results • Take home messages: • Micromedex is a popular resource • Ovid edges out PubMed • Lab tests are a popular context for resources • No correlation between context and resource • Nurses and pharmacists like the Micromeex infobutton for microbiology • Still needed: • Context/resource/user type interactions • Concept-oriented contexts • Questions: • How much is based on knowledge of resources? • What are the questions in each context?

  22. WebCIS User Questionnaires • Health resources page outside vs. inside WebCIS • Resources used, how often, how useful, anecdotes • Infobuttons used, how often, how useful, anecdotes • Other resources used

  23. WebCIS User Questionnaire Results 1055 users in January 300 randomly selected 250 with valid e-mail addresses surveyed 42 responded 21 attendings 19 housestaff 1 nurse 1 pharmacist

  24. WebCIS User Questionnaire Results How often do you access: Never<MonthMonthlyWeeklyDaily Attend 2 1 3 5 9 House 3 11 5 Nurse 1 Pharm 1 Attend 11 8 2 House 14 3 1 1 Nurse 1 Pharm 1 Resource Infobut

  25. WebCIS User Questionnaire Results Why do you access: Patient QsGenl KnowOther Attend 20 12 3 House 18 10 1 Nurse 1 Pharm 1 1 Attend 10 House 4 Nurse Pharm 1 Resource Infobut

  26. WebCIS User Questionnaire Results How do you access: NoneWC-LoginWC-MenuBothOther Attend 2 4 7 8 12 House 0 8 8 3 17 Nurse 1 Pharm 1 1 Resource

  27. WebCIS User Questionnaire Results What resources and how useful: Very Somewhat Not _ AHAHAH MMDX 17 17 0 2 1 0 Ovid 10 11 1 2 0 0 PubMed 12 7 2 3 0 0 Medline+ 4 4 0 1 1 0 FullText 8 7 2 2 0 0 Harrisons 7 4 3 9 1 0 PIER 2 4 3 1 1 0 Formulary 2 3 2 1 1 0 Health J 6 9 0 1 0 1 Science J 3 4 0 1 1 0 STATref 1 2 6 0 1 1 ICD9 1 2 5 0 0 1

  28. WebCIS User Questionnaire Results How useful are infobuttons: Very Somewhat Not _ AHAHAH Micro Org 3 3 5 1 2 0 Micro Sens 7 3 2 3 0 0 Pharmacy 6 1 0 2 0 0

  29. WebCIS User Questionnaire Results How does self-reporting correlate with log files? 100 80 60 40 20 0 A A A A A H A A A A A H H H A H H H H A A A H H H A H H H H H A H A A H A A H Never <Monthly Monthly Weekly Daily Health Resources N P

  30. WebCIS User Questionnaire Results How does self-reporting correlate with log files? 25 20 15 10 5 0 A H A H A P A H H A H H A A A H H H H H H A H H A A A H A H A N A A A A A H A H A Never <Monthly Monthly Weekly Daily Infobuttons

  31. WebCIS User Questionnaire Results • Anecdotes: • Avoid errors (drug interactions) • Identify correct doses • Identify pills • Prepare publications • Teaching • Understand abbreviations (sensitivity tests) • Other desired resources: • Up To Date • NEJM • MDConsult • Books

  32. WebCIS User Questionnaire Results • Resources are used and liked • Resources are used within WebCIS • Resources are used to answer patient-specific questions • Use of resources may be overestimated (can’t tell for sure) • Infobuttons are used and liked • Additional resources requested • Questions: • How knowledgeable are users about resources? • What are the context-specific questions

  33. Observational Studies • Connect portable usability lab to clinical workstation • Ask users to think aloud about information needs • Videotape screen and audiotape users • Transcribe tapes • Code transcripts, using tapes • Analysis of coding

  34. Portable Usability Lab User’s Workstation 75 foot cable Video Monitor Converter Controller Microphone Video Converter Headphones VCR Cassette Recorder

  35. Observational Results • Sites: • Medical /Surgical • CCU • Ambulatory Care • Number of Days: • 7 different days • Varying days of the week • Varying times of day • Total hours of tape: 15 hrs 29 minutes • Total Number of events: 154

  36. Observational Results Total Number of events: 154

  37. Observational Results Total Number by User Type

  38. Event Coding • Types of Events: • Foreground/Background • Implicit/Explicit • Patient/Institution/Administrative/Domain • Types of Resources • Human/Computer/Paper • Types of Outcomes • Success/Deferred/Failed

  39. Coding Event Type • Foreground • Specific information about patient management • Problem, Intervention, Comparison, Outcome • Background • Root question: who, what, where,when, why, how? • Related to health disorder or aspect of a disorder

  40. Coding Event Type

  41. Coding Event Type • Explicit: Subject expresses an information need • Implicit: Subject uses a resource other than WebCIS

  42. Coding Event Type

  43. Coding Event Type • Patient • A patient related need • Institution • An information need that is related to the infrastructure of the institution • Administrative • An institutional information need that is related to a specific patient • Domain • A domain related information need that is related to a specific patient

  44. Coding Event Type

  45. Coding Resources Used • Human • The subject sought information from a human interaction in person or via telephone • Computer • The subject sought information from a computer interaction either webCIS or other internet resource • Paper • The subject sought information from a paper resource – patient chart or text book

  46. Coding Event Type

  47. Coding Outcomes • Success • The subject sought and found the desired information • Deferred • The subject did not seek an answer • Failed • The subject sought, but did not find desired information

  48. Coding Outcomes

  49. Coding Events • Need to correlate across event types • Especially need to look at types of needs that were deferred or that failed • Good news: • Information needs exist and can be identified • Needed resources are computer-based • Bad news (good news for us): • Many needs failed or deferred • Solutions will require integration of patient data with information resources • Questions: • What are the triggers? • What resources can help?

More Related