1 / 15

MHV Usability Testing

MHV Usability Testing. Neale R. Chumbler, PhD, Jason Saleem, PhD, David Haggstrom, MD, MAS VA HSR&D CIEBP and Stroke QUERI. Acknowledgements. Alissa Russ, PhD, Josette Jones, PhD, Scott Russell, BS, Wanda Hines

maryfranco
Download Presentation

MHV Usability Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MHV Usability Testing Neale R. Chumbler, PhD, Jason Saleem, PhD, David Haggstrom, MD, MAS VA HSR&D CIEBP and Stroke QUERI

  2. Acknowledgements Alissa Russ, PhD, Josette Jones, PhD, Scott Russell, BS, Wanda Hines The research reported/outlined here was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (SHP 08-192). Dr. Chumbler is the Co-PI of the HSR&D CIEBP at Roudebush VAMC.

  3. Personal Health Records Holds great promise in promoting patient-centered care Very few studies have tested the efficacy or implementation of PHR systems. WSJ/Harris---75% of Americans would communicate with their physicians if given means to do so 2nd survey --- 60% of patients would look up test results & track medication use if records were available. Kaelber et al., 2008

  4. Personal Health Records • VA’s PHR, MHV, has reported satisfaction survey results. • Many positive findings • Room for improvement in: • Ease of navigation • Need for more usability testing

  5. AIMS Identify factors impacting patient use of MHV Explore how to engage patients in the best use and navigation of MHV Usability testing is congruent with the MHV Evaluation framework foci

  6. MHV Evaluation Framework • MHV framework has adopted the RE-AIM Performance Framework as a tool to organize and synthesize work on MHV. • Our work is congruent: • Functionality • Navigation • Search

  7. What is Usability • Usability measures the quality of a user’s experience when interacting with a product or system • Its not a single one-dimensional property. It has been associated with five attributes: • Learnability Low error rate • Efficiency Satisfaction • Memorability

  8. What is Usability Testing • Usability testing is carrying out experiments to find out specific information about a design. • Evaluating a single design • Comparing two or more designs • Determining if a design works in real life (ideally done prior to full implementation) • Meant for rapid feedback to design team so changes can be made iteratively throughout the design cycle.

  9. Factors Influencing Implementation of MHV • Phase I --- semi-structured interviews with 8 patients who previously used MHV • How often do you use MHV? • Are their specific things that prevent you form using MHV more frequently? • What type of things do you use MHV for? • Have you experienced frustration in finding information in MHV? • Are there specific functions in MHV that you find difficult to use?

  10. Usability Test of MHV • HCI/IT lab…n = 24…simulate a typical setting that a patient would access MHV. • Qualitative interviews helped inform the design of the usability scenarios • For example… • Prescription refill (10 minutes)

  11. Usability Metrics • Quantitative --- • 1) Efficiency (time to complete scenarios) through time stamped Moraedigitial video recordings 2) Error rates (observed deviations from intended use of MHV) 3) Satisfaction (7-point Likert Scale)

  12. Initial Usability Results (In Progress) • Rx refill time on task • No significant difference in actual time task performance (mean = 209 sec) for 1 Rx refill and target task performance (240 sec; 4 min) for first Rx refill task • Actual task performance (mean = 102 sec) for same Rx refill task after second run was significantly lower than target task performance (240 sec; 4 min); p=.007

  13. Initial Usability Results (In Progress) • Participants rated two usability statements related to error messages and recovery from mistakes significantly lower than target usability levels (target = 5 on a 7 point Likert-type scale where 1 = strongly disagree and 7 = strongly agree) • The system gives error messages that clearly tell me how to fix problems. (mean rating = 3.2), p-value = .001 • Whenever I make a mistake using the system, I recover easily and quickly. (mean rating = 3.5), p-value = .001

  14. Lessons Learned • Usability testing can provide important and prompt design feedback to increase user satisfaction through participation of actual end users. • Collaboration with central office partners (Kim Nazi) increases value of work to VA. • A priori usability targets, like a priori hypotheses, are of greatest value from study design standpoint. • Basic steps in application design should not be overlooked • e.g., log-in functions

  15. Plans for Moving Ahead • Consider usability scenarios for disease-specific conditions • e.g., cancer, stroke, diabetes • Further integrate MHV usability research agenda with work of MHV design team

More Related