Download
1 / 18
180 likes | 191 Views
This usability test plan outlines the purpose, objectives, subject profile, test design, monitoring methods, evaluation measures, and report presentation for conducting usability tests on software applications. It covers different test types such as comparison, pilot, field, beta, and acceptance tests, detailing criteria for evaluating usability and performance. The plan emphasizes clear test objectives, subject demographics, test groups, monitoring guidelines, and data collection measures. It also includes performance and preference metrics to assess user satisfaction and task completion efficiency.
E N D
Larson 6 Usability Test
Test Plan • Purpose • Problem statement/test objectives • Subject profile • Test design • Monitor • Evaluation measures (data to be collected) • Report contents and presentation
1. Purpose • Which type of test? • Comparison test—Which is “better?” • Pilot test—Test the test procedure • Field test—Can real users really benefit by using the system • Beta test—Users help debug the system • Acceptance test—Validate that the system satisfy the requirements? • What do we hope to learn?
2. Test objectives • Avoid unfocused and vague problem statements • Examples of poor objectives • Is the current product usable? • Is the product ready for release or does it need more work? • Examples of good objectives • Do the screens reflect the content model? • Is help easier to access via a “hot key” or via a mouse selection?
3. Subject profile • General computer experience • Range: none to two years • Education: • Range: 10% high school, 60 % college, 20% masters 10% Phd • Age • Range: 85% ages 20-50, 15% other • Gender • Education Major: 0% CS, 100% other
4. Test design • Detailed plan for conducting the test • Groups of subjects, e.g., • Group A: Mary, Fred, Sam, Jose • Group B: Sue, Ron, Bob, Sally • Tests • Group A does test 1 • Group B does test 2 • Subjects read from printed instruction script
Instruction script • Orientation • Introduce yourself • Offer refreshments • Explain why subjects are here • To test a new application • Stress that the subject is not being tested • Describe the equipment the subject will use • Explain that session is being recorded • Nondisclosure form • Lists of tasks to perform. For each task: • Provide data user needs to perform the task • Do not explain what commands to widgets the user should use. • Written debriefing questionnaire
In class exercise • Write Test Instruction Script for your application
5. Monitor the test • Try to be objective • Enable rather than lead the subject • Don’t act too knowledgeable • Don’t jump to conclusions • Let the subject struggle
6. Evaluation measures • Performance • Data collected during the test • Preference • Opinions collected during debreafing
Two types of usability tests Is the application useful? Is the application enjoyable? Performance Preference Measure users’ likes and dislikes Measure what the user actually accomplished Validate that users enjoyed the application and will uses it again Validate that users achieved success
Caller Task Measure Typical Criteria Locate “Modern Girl” by Sleater-Kinney Complete successfully Less than 40 seconds Purchase “Modern Girl” Complete successfully Less than 2 minutes Performance Metrics
Caller Task Measure Typical Criteria Locate “Modern Girl” by Sleater-Kinney Complete successfully Less than 40 seconds Purchase “Modern Girl” Complete successfully Less than 2 minutes Performance Metrics Write performance metrics for your application
Question Typical Criteria On a scale from 1 to 10, rate the help facility. The average caller score is greater than 7. On a scale from 1 to 10, rate the understandability of the error messages The average caller score is greater than 7. Would you recommend using this applications to a friend? Over 75% of callers respond by saying “yes.” What would you be willing to pay to use this voice-enabled application? Over 75% of callers indicate that they are willing to pay $1.00 or more per use. Preference Metrics
Question Typical Criteria On a scale from 1 to 10, rate the help facility. The average caller score is greater than 7. On a scale from 1 to 10, rate the understandability of the synthesized voice. The average caller score is greater than 7. Would you recommend using this applications to a friend? Over 75% of callers respond by saying “yes.” What would you be willing to pay to use this voice-enabled application? Over 75% of callers indicate that they are willing to pay $1.00 or more per use. Preference Metrics Write preference metrics for your application
Preference Metrics (Open-ended Questions) • What did you like the best about the voice-enabled application? Do not change these features • What did you like the least about the voice-enabled application? Consider changing these features • What new features would you like to have added? Consider adding these features in this or a later release. • What features do you think you will never use? Consider deleting these features. • Do you have any other comments and suggestions? Pay attention to these responses. Callers frequently suggest very useful ideas.
Lab Assignment • Write a text plan • Write the usability test design • Conduct the usability test • Record test data and summarize each of the performance and preference metrics • Place all of your documentation into your lab log and send URL to instructor
