Usability Testing

Usability Testing

Today in Class • Finish Presentation from last week on User Experience • Usability Testing Presentation • Discuss Homework assignment

Overview • What is Usability Testing • How Usability Testing Works • How to Conduct a Usability Test • Develop a Usability Test Plan • Post-test report writing and recommendations • Testing Methods: Selecting the Right Method for a Project

Usability Testing… A technique used to evaluate a product by testing it with representative users Complete typical tasks while observers watch, listen and takes notes What is it?

Goals of Usability Testing • Identify any usability problems, collect quantitative data on participants' performance and determine participant's satisfaction with the product • Provides feedback during the design/development process

Characteristics of Usability Testing • Defined objectives • Uses real users • Real tasks • Early and iterative testing

Characteristics of Usability Testing Should be clarified well in advance of the actual testing Allow web developers to choose: Test participants Test methods User tasks Defined Objectives

Characteristics of Usability Testing Similar characteristics “If the participants are more experienced than actual users, you may miss problems that will cause the product to fail in the marketplace. If the participants are less experienced than actual users, you may be led to make changes that aren’t improvements for real users.” Allows users’ needs, rather than the designers’ preferences and biases Real users

Characteristics of Usability Testing Represent the tasks users will actually perform Real tasks

Characteristics of Usability Testing Begin early and continue to the final product Helps drive design choices and make modifications Early and Iterative testing

Data Generated from Testing • Behavior/Objective • Opinions • Objective Data • Subjective Data • Qualitative or Quantitative

Value of Usability Testing • Minimizes costs • Minimize risk • Increase revenue, product sales, and brand loyalty • Acquire a competitive edge • Create a historical record of usability benchmarks for future release

Testing Basics: 5-Step Process

Step 1: Plan & Prepare • Develop a test plan • Allows you to communicate your goals with the client & present expectations

Types of Scenarios • Goal- or Task-Based Scenarios • Elaborated Scenarios • Full Scale Task Scenarios

Using Scenarios in Website Design • Write down 10 to 30 of the most common reasons that users have for visiting or tasks that users want to do • Scenarios can also work together with personas by serving as the stories behind why the particular persona would come to your website • Focus on users and their tasks rather than on your site's organization and internal structure

Using Scenarios in Usability Testing • Limit your test to 10 to 12 taks • Usability testing scenarios should not include any information about how to accomplish a task • Write down how to accomplish the task

Deciding What Questions to Ask • Chapter 3 (16:18)

Making a Task List • Chapter 4 (7:19)

Step 2: Find Participants • Depends on participants • User profile that details characteristics of potential users of the website • Work experience. • General experience with the website • Experience with similar products • Test your test plan beforehand with co-workers or friends that have an acceptable degree of web user experience

Usability Testing Participants • Chapter 2 (20:14)

Step 3: Conduct the Session • Introduce yourself, explain the process to the user • Inform users that they be asked to perform a set of pre-defined tasks • Make the user feel comfortable • Speak only to give a new task and take notes during the process

Step 3: Conduct the Session • Prepare a short summary of the session and the results • Outline specific problem areas and any unexpected results • Include any personal observations

Step 3: Conduct the Session • Collect basic data: • Could the user complete the task? • Did they need help? • Track how much time it took them • Note any stumbling blocks (problems/obstacles) • Overall observations, commentary • Debrief the user, allow user to speak their mind

Step 3: Conduct the Session Post-Test Survey: • Prepare a survey online or in paper form • Questions should include • Graphics • Logic • Content • Navigation, • Overall satisfaction • Gather data about overall effectiveness of the site in relation to the goals of each task

Questions you might ask • Have you visited this site before?Previous familiarization with a site can skew first impressions. • What do you think the purpose of this site is? (ie. selling, informing, entertainment, etc)If they think it is a selling site, but it is actually a content information site, question what made them think the purpose was different than it really is. • Who do you think the intended audience is?You know your targeted demographic, but perhaps they noticed clues that would leave them to believe a completely different audience was intended. • Could you find what you were looking for?You want to know if everything was there the user expected or if there was something he or she thought they’d find but didn’t • Was it easy to get to the home page from the page you started on?If the user expresses trouble getting to the home page, reassess your navigation structure or find out where they expected to find a link to the home page but didn’t. • Was there something missing you were expecting to see?For prompting, you can ask about more text, more images, a FAQ, a question answered, etc.

Questions you might ask • Could you tell what the page was about?If they sound confused, ask specifically what they thought it was about, and what those indicators were. • Was anything too obtrusive?Particularly important if you use pop-up or pop-under ads, an in-your-face style of ad placement, use of flash, etc. • Was anything too well hidden?If you noticed he or she seemed to be hunting around the site, prompt with this. • Problems or kudos on the color scheme?Too flashy? Too bland? Just right? • Easy to read (both font style and size)?Was the font size too large or small? Was the chosen font difficult to read or in a color that made the text not as readable as it could have been? • How did you find the layout of the site?Was everything organized well and set out as expected? If the layout is not a usual style, question that experience as well.

Questions you might ask • How intuitive and helpful is the navigation system?If you are using any kind of JavaScript or floating menus, this question is crucial to find out if there were any problems from a visitor’s perspective. Oftentimes there are. • Did you notice… (advertising, newsletter signup, video, search box… etc)Any other elements of the site or design you need to ask? • What would encourage you to return to this site in the future?Was there something that could have been added to increase the return visitor rate? • Name your three favorite things about the site, and your three least favoriteThis usually can bring up the unexpected things about your site that either endear people to it, or make them more likely to bounce quickly. • If you could change one thing on the site, whether it is major or minor, what would be at the top of the to do list?You will get all kinds of responses, however it can help you gauge what people see as the most important things that hindered their experience. But if multiple people all say how annoying a single element is, such as the auto-play video you placed on the home page is, chances are good you should probably remove it.

Moderating a Session • Chapter 7 (8:44)

Step 4: Analyze Results • Compile and summarize data • Transfer handwritten notes to computer • Write your reports while they are fresh in your mind, • Create a summary after testing is complete, into a table that shows the results of each test, include problem areas, comments and user feedback from the survey

Step 4: Analyze Results • Identify: • Difficulties and problem areas • Why there was difficulty or the source of any problems • Any specific task-oriented issues

Step 5: Make Recommendations • Compile and recommend • Gather all your compiled information and translate into recommendations • Concentrate on high-level functionality first • Then focus on recommendations for improved user experience • Determine the implementation plan • Write up a formal report

Analyzing and Reporting Results • Chapter 9 (8:26)

Usability Testing Tools

Paper and Pencil • Pros • Fast • Cheap • Cons • Early design stage testing only • Not for use in testing interaction

Silverback • http://silverbackapp.com/ • Pros • Reasonable Price $69.99 • Cons • For Mac

Google Analytics • Free • Easy to Use • Identifies: • User behaviors • Trends • Red-flags

Google Content Experiments • Originally called Google Website Optimizer • Free Multivariate testing tool • Pros • Free • Cons • Does not identify the “why”

In-class Activity • For the following task-based questions, please begin at the URL provided. Each student who is not performing the task will take notes on the steps used to find the answer. We are not interested in the answers themselves but rather the steps that the user took to find them. • Please note: It is possible that a task cannot be completed.

Internet Movie Database • http://www.imdb.com/ • Use case: Find out which films Denzel Washington was in during the 90s. Pick pick one of the titles and find out if it is available on DVD or not.

U of Arts • http://www.uarts.edu/ • Use case: You are an artist and want to learn how to present your work online and what resources are available to you. The U of Arts has an upcoming lecture geared toward this. You want to know how much the lecture cost to attend and who you should contact to RSVP for the event.

Temple University • http://www.temple.edu/clait/ • Use case: You are a professor in the College of Liberal Arts at Temple University. You are planning to hold a lecture series. You need to find a room to hold your lecture series in. You are expecting to have no more than 75 people attend each session. Once you find a room that fits your needs, who do you contact to reserve the room?

Department of Psychology • www.temple.edu/psychology • Use case: You are a graduate student looking to apply to the Department of Psychology at Temple University’s graduate program. You need to now what are their requirements for admission. Once you have found the requirements, when is the application deadline?

U of Arts • http://www.uarts.edu/ • Use case:You are a professor teaching a course in the fall. You need to know when your final grades should be handed in. What day should all grades be submitted?

Usability Testing