Usability Testing with People with Disabilities

Usability Testing with People with Disabilities Kate Walser Whitney Quesenbery

Hi Kate Interaction design, usability testing and slipping accessibility under the radar. Whitney Users’ stories, plain language, discovered accessibility as a right in elections What about you?

What do you want to learn today? Do you all have experience running at least one practice usability test? Any specific questions or things you want to learn?

Here’s what we planned Recap of how to run a usability test Testing with people with disabilities Practice tests Debrief and applying what you learned

Schedule

Review: What is usability testing 1

Usability testing lets you see behavior Observe • People like the audience using the site rather than asking them about it • How easily they meet their goals • What causes confusion or problems Inform design • Lets you confirm or challenge assumptions about the design. • Gives you the opportunity to fix problems. Informal usability testing of voting materials at the Farmer's Market in Olympia, Washington

How easily can we learn from users? We learn from users by watching them use the site or other materials. • Don’t explain or demo. • Watch what they do. • Listen to their comments. • Take their problems seriously. Dona Vitale conducting a usability test in Chicago, 2005

The moderator runs the session • Prepare and anticipate problems • Observe without influencing outcomes • Make participants feel valued • Impartial, unbiased, observing • No teaching! • Listen and watch • Open-ended questions: Why?How? What were you doing?

Observers and note-takers Watch (and listen) quietly for whether the user • Is confused by anything, or has to re-read information • Has to correct mistakes, or backtrack to correct naviation Take good notes • Write down what you see and hear. Don’t “translate” or put words in the user’s mouth • Be specific. Not "he's confused." But "he said he doesn't know what APR means.” • Don't infer the user's reasons for doing something. • Don't solve problems while taking notes. That's for later. It will take all your concentration to note what is happening.

Practice tests(20 minutes each) 6

Moderating and observing • Conduct sessions the same way. • Ask the participant to “pause” the AT or lower the volume as they may have thoughts they want to share. • Remember that the AT becomes a large component of the UX • Avoid interrupting the user as they use the AT • Remember to watch the cursor, not the mouse.

Get into teams If you are the moderator, use your practice test. If you are an observer, think about What you will watch for Where to sit so you can see, but won’t distract the participant. Challenges observing this person’s AT or accessibility setup. Remember: focus on what happens, not analysis or recommendations

Practice Test 1 FILL IN TASK HERE

How did that go? What felt awkward? What worked well? Anything surprise you?

Summary: Usability testing with people with disabilities 7

Consider how different AT and disabilities affect usability test logistics Recruiting Participants Scenarios and handouts Incentives A place to test

Recruiting Allow more time to find participants • Screener + disabilities Look for community groups and ask • Meetup, Facebook, Twitter Use specialized recruiting services • KnowbilityAccessWorks portal Work with outreach groups • Advocacy groups • Employment non-profit groups • Colleges and universities

Environment & Setup Location becomes even more critical • Is it reachable by public transportation? • Is the building itself accessible? • Are the signs to find the room accessible? Room setup matters • Is there space to navigate with wheelchairs or walking devices? Is there space for a service animal to lie down near its human? • Are tables or desks wheelchair accessible? • If visually impaired, can the person sit close enough to the monitor to view the screen? Can you mirror the screen on another monitor for the moderator to view? • How close are other workstations if screen readers will be used?

For in-person tests, explain setup and ask about accommodations Explain what will happen and ask about specifics • “We usually provide handouts for users to read that describe the scenario. What’s the best way to provide that to you?” Describe the setup, and include accessibility specifics in your confirmation emails • “We’re on the 3rd floor. There’s a ramp to the right of the front doors. The doors automatically open. There’s an elevator bay in the lobby to get to our floor.” Ask if they will bring anything • Service animal (would they like a rest or water stop for the animal?) If refreshments will be served, ask about dietary restrictions

Schedule Add time between sessions for • Setup time • Moderator break Dry runs are critical, preferrably with a representative user or else, with someone trying to reflect the different considerations • Online: Try downloading / opening the tool, going to the scenario, talking with the moderator with AT on • In-person: Try going from closest transportation spot to location, into building to designated location, taking / using handouts

Tips for Moderating and Observing Act like you normally would • Treat them like you would any other colleague or friend • Ask first before assuming they need help Learn the AT • Practice listening to JAWS or other assistive technology while someone is talkingAsk first before assuming they need help • Know when the AT will affect how you interact with the participant Look for general usability problems as well as those made worse by poor accessibility or interaction with the AT • Missing information, problems navigating can reveal problems for everyone • Look for issues caused by the presentation

Summary: Testing with People with Disabilities Consider logistics, location, and setup Allow more time for recruiting and between sessions Do a dry run and practice observing

Want to learn about remote usability testing? Next session Running usability tests remotely When you work directly with the participant And when they work alone

Whitney QuesenberyCenter for Civic Designwhitney@centerforcivicdesign.org908-617-1122 Kate Walser CX Insights kwalser@cxinsights.com571-281-2626

Usability Testing with People with Disabilities