1 / 19

Long-term Usability Testing for Public Health Information Technology: BioSense 2.0

Long-term Usability Testing for Public Health Information Technology: BioSense 2.0. Amanda Recker Jamie Pina, MSPH, PhD Barbara L. Massoudi, MPH, PhD. RTI International 2013 International Symposium on Human Factors and Ergonomics in Health Care: Advancing the Cause March 11, 2013.

yama
Download Presentation

Long-term Usability Testing for Public Health Information Technology: BioSense 2.0

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Long-term Usability Testing for Public Health Information Technology:BioSense 2.0 Amanda Recker Jamie Pina, MSPH, PhD Barbara L. Massoudi, MPH, PhD RTI International 2013 International Symposium on Human Factors and Ergonomics in Health Care: Advancing the Cause March 11, 2013

  2. BioSense 1.0: Web-based syndromic surveillance • Mandated in the Public Health Security and Bioterrorism (BT) Preparedness and Response Act of 2002 • Nationwide integrated system for early detection and assessment of potential BT-related illness • 2003 Funding provided by Congress to CDC • Development of BioSense infrastructure started, initial focus on: • VA and DoD • Direct reporting to CDC of detailed clinical data by civilian hospitals • Began soliciting more limited data from health departments (HDs) that had already established automated systems for ED-based syndromic surveillance • By 2007, 8 state/local HDs connected

  3. Recommendations from Prior Evaluations • Strengthen state and local public health engagement • Enhance state/local HD syndromic surveillance capacity • Increase participation of state/local HD syndromic surveillance systems (improve coverage) • Share data with HDs from hospitals reporting directly to CDC • Share governance with public health community • Leverage investments in electronic health records (EHRs) • GAO, 2008: Adopt an “open, distributed computing model” • Improve utility of the data and data sources • Preparedness role: Greater “all hazards” emphasis • Expand uses for broader spectrum of public health concerns

  4. BioSense 2.0: Timeline • June 2010: Redesign begins • November 2011: Opened for business • November 2011 – June 2013: Onboarding new jurisdictions • 35 jurisdictions signed the Data Use Agreement (DUA) • 17 fully onboarded • April 2012: Retired BioSense 1.0

  5. BioSense 2.0: Approach • Shift from a need-to-know to a need-to-share and co-create approach • User-centered design • Stakeholders engaged in every step of the redesign • HDs fully control “their data” at the level of granularity they choose • More options for data sharing with other jurisdictions and CDC • Alignment with ONC and Meaningful Use • Agreed-upon core syndromic surveillance data elements • Collaborations with public health professional associations • Funding to states: Meaningful Use syndromic surveillance adoption, build capacity, join BioSense 2.0 • Cloud technology: distributed, easy to adopt, cost effective, secure

  6. Application Home Page

  7. Encephalitis, Meningitis, WNV = CNS Inflammatory Disease

  8. Why Long-term Usability Testing? • Long user-centered design lifecycle • Expectation management • Stakeholder ownership • BioSense 2.0 continuously changing and growing • Longitudinal usability testing • User satisfaction • Efficiency • Functions not changing: building a query, viewing results, analyzing the data, sending and saving information • Two approaches to testing

  9. 1. User-Centered Design • Qualitative data collection methods • The user knows best • Test the right participants • Broad range of public health professionals • Public health generalist – less sophisticated users • Syndromic surveillance epidemiologist – more sophisticated users

  10. 2. Activity-Centered Design • Quantitative data collection methods • Behavior vs. opinions • Based on empirical data • Time-on-task analysis • Mouse-click analysis • Pathway analysis

  11. Basic Activities in Interaction Design • Establish requirements • Design alternatives • Develop prototype • Conduct evaluation (Rogers, Sharp, & Preece, 2011) (Rogers, Sharp, & Preece, 2011)

  12. How to Choose Users • Interact directly with the system • Epidemiologists, state and local public healthprofessionals • Manage direct users – decision-makers • Public health directors • Use similar syndromic applications

  13. Generating Design Alternatives • Activity-centered design • Morae software • Scenarios and tasks • Closed/open question response • Time on tasks • Mouse-click analysis • Pathway analysis • Critical incidents • User-centered design • Open question response • Focus groups • Expectation testing • SUS • Expert evaluation • Usability heuristics (Nielsen)

  14. Scenarios and Tasks Scenario: “Over dinner at [a public health conference] an argument has erupted, but luckily as a BioSense 2.0 user you can settle this dispute. Health authorities in Virginia suspect that the flu season was more severe than it was in Michigan.” Task: “Please determine which state, Virginia or Michigan, had more cases of influenza-like illness (ILI) starting in October 1, 2010 through March 1, 2011.”

  15. Pathway Efficiency Analysis • Time on task • Mouse-click analysis • Pathway analysis

  16. Design Alternatives

  17. How to Choose Among These Alternatives? • If one person says something is a problem, do you change the design? • Resolved conflicting alternatives • Conducted feasibility analysis • In the end…relied on face validity

  18. What Happens After Design Changes? • User training through webinars and videos • Expert user testing • Focus group sessions • Continually comparing SUS scores

  19. Future Activities • Incorporate eye tracking into testing protocols • Conduct on-site testing and evaluation • Task analyses of routine versus event surveillance • Information models for routine and event surveillance

More Related