National Efforts for Measuring EHR Usability and Adoption: Issues, Opportunities and Challenges

1. National Efforts for Measuring EHR Usability and Adoption: Issues, Opportunities and Challenges Andre W. Kushniruka,c,, Johanna Kaipiob, Marko Nieminenb, Christian Nohra a School of Health Information Science, University of Victoria andrek@uvic.ca b Department of Computer Science and Engineering, Aalto University, Finland c Department of Development and Planning, Aalborg University, Denmark

2. Issues in Evaluating Electronic Record Systems Usability in Canada • Widespread adoption of electronic health records (EHRs) and related IT is well underway in Canada • Provincial and federal programs fund up to 70% cost • However: • Evidence about usability issues from multiple types of users (e.g. physicians, nurses, pharmacists) needed • Evidence from multiple locations and regions (different hospital settings and uses) needed • Evidence about usability of different vendor systems needed • Need to share such information about system usability • At local, regional, national and international levels • For improving system usability, selection, procurement and regional deployment

3. Issues • Approaches to implementation and type of systems implemented have varied greatly • Simply deploying EHRs and health IT does not guarantee that end users find them effective and usable • Can do small-scale usability studies, which are necessary and needed • However, need approaches to collect larger-scale data on use of specific EHRS and end user perception of their effectiveness and usability

4. Approach in Canada • In Canada, to assess usability of EHRs in the large-scale, two main approaches: • EHR Rating Websites • Provincial websites that allow end users of EHRs (e.g. physicians) to post their ratings of particular vendor products (i.e. EHRs), which can be reviewed by other physicians considering purchasing one of those systems • the National Physician Survey (NPS) • a lengthy questionnaire sent out to all physicians, focused on a wide range of aspects of medical practice in Canada, including use of EHRs and other technologies

5. National Physician Survey (NPS) • Study Aims: • To characterize key aspects of physician experience in Canada (e.g. type of practice, type of patients etc.) • Includes scales about use of health information technology (HIT) • Methods: • Questionnaire mailed or emailed out to Canadian physicians • Sample: Last version given mailed out (email and regular mail) to all Canadian physicians • 12,076 replied (response rate of 18%)

6. Examples of NPS Questions Related to HIT • Thinking about your MAIN patient care setting, which of these describes your record keeping system? • Paper • Paper-electronic • Electronic only • Where do you access electronic records for your patient? • If you access electronic records in various locations, are the records electronically connected to allow for access to the same record from different locations?

9. Limitations • Few questions specifically about usability/user experience in NPS • Not offered every year • Does not provide vendor specific information about usability or adoption • National survey needs to be expanded to include greater emphasis on user experience • Recommendations made to government

10. EHR Rating Sites in Canada • Emergence of websites specifically about user experience with specific vendor products and solutions • Part of provincial and regional programs to expand use and adoption of EHRs • Allows for entry of subjective ratings of specific vendor products by physicians • Can guide physicians in buying (procuring) systems as they can check the ratings made by their colleagues • Ontario MD website is a good example • Allows physician users to rate EHRs (including rating of usability) • Physicians interested in buying and EHR can check the ratings made by other physicians of that system

13. Future Directions • Canada needs a mechanism for end users to enter information about usability and their experience in using specific health information systems • NPS should be expanded to include such questions • Need to expand sites that allow for rating of specific systems and to make the ratings richer (not just rating of overall usability on a 1-5 scale) – including rating of the dimensions of usability: • E.g. scales for effectiveness, efficiency, learnability, enjoyability and safety of systems (along with free text descriptions – i.e. more text box in the questionnaire)

14. Further Directions in Canada • Large-scale usability questionnaires could be complemented with qualitative usability studies conducted across regions in Canada • This could include collection of subjective user ratings and objective measures of usability • Using rapid low-cost usability engineering processes • Could publish information about vendor product usability based on this

15. Additional Issues • If usability (in addition to adoption) data can be collected nationally, what can be done with it? Who will analyze it? Where will recommendations go to?