Got Five?

1. Got Five? Effective User Testing With $5 and 5 People Liz Norell enorell@twu.edu Director of Web Communication Texas Woman’s University Denton, Texas July 19, 2005

2. But first… Materials, handouts and slides online: www.twu.edu/usertesting/

3. Texas Woman’s University • Fall 2005 enrollment projected at 11,450 • Annual enrollment growth at about 6% • One of the smaller TX state universities • Four campuses: Denton, Houston, two in Dallas • FTE approximately 650

4. TWU Web specifics… • One Web director; approximately 125 staff (on four campuses) who maintain departmental sites • Migrating to new Web templates; completed about 75% in last 18 months

5. What is user testing? • Trying to determine how usable your Web site is – specifically: • Can people find what they’re looking for? • How quickly can they navigate? • Have the Web developers and designers anticipated user behavior?

6. Who should be the testers? • New or current students • Prospective students • Faculty and staff members • Recent graduates • General public

7. What does the tester do? • Complete a series of Web tasks • Agree to be videotaped and/or observed while testing • As much as possible, mirror their genuine browsing style away from the observers (e.g., scatterlinking, giving up, etc.)

8. How many testers? • Each iteration should comprise no more than four or five testers • Focus on multiple iterations, not more testers • Little or no value to using more than five testers per iteration

9. Warnings • Avoid: • Talking to the participants once they begin • Gasping (at least, not audibly) • Coaching the testers • Answering their questions • Drawing attention to yourself

10. But seriously, with $5? • Students – particularly brand-new students – are usually eager to provide feedback on the Web • Minimal enticement is needed to recruit them – things that have worked for us: • Free water (chilled helps) • Free pens • T-shirts

11. A look at the Liz Norell approach to user testing(if you will)

12. Step 1: What needs tested? • Determine what content areas you want to test • Define the scope • The University Web as a whole • One department • One campus

13. Step 1: Suggestions • Work with administrators on identifying content areas and the scope • Solicit input from those whose sites are most likely to be involved in the test • Give opportunities for feedback, even (sometimes) if you won’t use that feedback

14. Admissions: How to apply, etc. Financial aid: Application deadlines, emergency loans Degree programs: Admissions requirements Faculty profiles: Research interests, CV material Registrar/registration: Last day to drop, payment deadlines, how to register Student services: Library, dining services, residence halls Step 1: TWU

15. Step 2: Draft questions • Write questions that require students to look at key content areas • Make sure they’re answerable • Circulate among stakeholders (including department chairs/directors AND Web staff) for comments

16. Step 2: Suggestions • Be specific in your questions • Avoid yes/no questions • Ask obscure questions • Solicit feedback on wording, NOT content, of the questionnaire • Test the questions on a colleague or friend

17. Step 2: TWU • What is the deadline for applying for fall 2005 financial aid? • The Student Center organizes a book club each month. What is the book for April? • Dr. Timothy Hoye is a faculty member in Government on the Denton campus. Where did he get his Ph.D. degree?

18. Step 3: Schedule testing • Ideally, schedule three iterations of testing approximately 1-2 weeks apart • Realistically, ensure there’s enough time between iterations to make adjustments • Ensure your IT staff can accommodate your testing needs in a lab or classroom

19. Step 3: Suggestions • Pick a time when students aren’t rushed • Orientation (after registration) is ideal • Avoid finals • Test on a Monday or Tuesday • Interest and willingness wanes later in the week

20. Step 3: TWU • Transfer and freshman orientations—immediately after students register • Student computer labs during the semester • Early in the week • Just before lunch

21. Step 4: Personnel • A successful user testing session can be done with just one staff person (i.e., you) • Ideally, recruit one or two student or staff assistants to assist • They recruit students • You observe the tests

22. Step 4: Suggestions • Students are a great asset as assistants during a user test • They present a familiar face • Students are more likely to respond favorably to another student than an administrator

23. Step 4: TWU • First complete user test was done with one person only • Subsequent testing has always included at least one other person • At smaller campuses, one person is plenty

24. Step 5: Where to test? Controlled environment versus Representative environment

25. Step 5: Suggestions • More open, public areas more closely mirror reality for most students • Browsing college Web sites without input from others may not be realistic for your prospective students • Ensure you’ll be able to offer some privacy for students who want it

26. Step 5: TWU • I’ve tried both controlled and public areas, and much prefer the public ones • More accurate representations of reality for our students • Students seem more comfortable, less “observed” in a public setting • Computer labs have worked very well for us

27. Step 6: Recruiting testers • Signage is critical • Approach people who don’t appear rushed • Don’t be a bother … but, don’t be afraid to approach people • Be clear you need their help • Don’t recruit more than two students for every one observer

28. Step 6: Suggestions • Here’s where you spend your five bucks: • Free water • Free pens • Make eye contact before you approach • Don’t be discouraged if it takes awhile to get five people – it is, after all, only five people

29. Step 6: TWU • Students more effective for us than staff members as recruiters • Students love free stuff • Students want to help • And, they WILL help if they can

30. Step 7: Conducting the test • BE QUIET. • Be QUIET. • Don’t say anything. • SHUT UP. • Don’t gasp. • Don’t coach.

31. Step 7: cont’d • Now that that’s out of the way … • Make sure to give all appropriate caveats • Alert testers that you’ll be watching • Let them start in their own time • Don’t rush them or act impatiently • Quietly track their clicks by writing notes or videotaping

32. Step 7: Suggestions • Try out your role as tester ahead of time • Only intervene if an enormous amount of time has lapsed • Take the best notes your handwriting will allow • Duplicate effort between two people if you have the staff to do it.

33. Step 7: TWU • My first user test was an exceedingly humbling exercise • Giving testers a lot of reassurance seems to have helped

34. Step 8: Completing the test • Remind testers of optional information at the front of the testing document • Thank them for participating • Provide any freebie giveaways

35. Step 8: Suggestions • Give testers your business card • Respect their time – make additional questions optional • Keep it casual

36. Step 8: TWU • Low-cost giveaways (especially water) have been more popular than more expensive ones • If your budget allows, cameras are a good giveaway at orientations • Students respond to gratitude

37. A quick recap…

38. Step 1: What needs tested?

39. Step 2: Draft questions

41. Step 3: Schedule testing

42. Step 4: Personnel

43. Step 5: Where to test?

44. Step 6: Recruiting students

45. Step 7: Conducting the test

47. Step 8: Completing the test

49. Using the results

50. Transcribing your notes • Rewrite notes into discernible navigation paths • Do it quickly • Can you spot any trends?