Implementation of Usability Testing in Software Development Companies

1. Implementation of Usability Testing in Software Development Companies Murat BİŞİRGEN Mehmet GÖKTÜRK May 2011 1

2. Logo Business Solutions since 1984 2

3. 1984 Establishment 1986 Business App.s - DOS 1996 Business App.s - Windows 2000 First Software Co. in IMKB (Ist.Stock Exch.) 2000 Moved to Gebze Campus 2005 Innovative Open Architecture Award - IBM 2006 Business App.s – Java 2007 Unity2, Tiger2, GO, START 2009 Innovative Technology Award - TUSIAD, TUBITAK, TTGV 2010 Tiger Enterprise, Tiger Plus, GO Plus 150 Employees 170.000 Customers 1.200.000 Users 500 Resellers 25 Countries 15 Languages 60% Market Share (customers) LOGO by Years and Numbers 3

4. Leader in R&D League among local software companies and 14th among all Turkish companies Partnership with global leaders Microsoft, IBM, Oracle, Intel, ... Strategic partnership with local industry leaders Turkcell, Garanti, TTNET, ... Campus in Gebze, offices in Ankara and İzmir Authorized resellers in EMEA countries ERP solutions focused to customers ranging from 1-man entrepreneurships up to corporate enterprises HR solutions including payroll calculations and a large set of contemporary HR processes Vertical market solutions Development and integration tools for solution partners Facts 4

5. 5

6. Motivation 6

7. Introduction • Look at the product’s current performance • Conduct a task analysis • Create a “first draft” of the product • Create a set of realistic tasks • Set up Usability Test lab • Have observers measure and note difficulties and mistakes • Summarize findings/Make suggestions for improvement 7

8. Key Subjects • Usability • Usability Studies • Corporate Environment • Practical Suggestions 8

9. Usability “The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction.” (ISO 9241) 9

10. Usability Studies • “The evaluation of a product’s usability through direct observation of user behavior during a structured task.” • Key concepts: • User • User Behavior • Direct Observation • Structured Tasks 10

11. Test Participants A sample of people with a similar profile from the product’s target users. 11

12. Sample Characteristics for Participant: • A person at a company responsible for a certain job • Between ages of 18 and 60 • Having the same number of female and male • (Not) A customer of a certain company • Using a computer at least 15 hours a week • Having at least 2 years experience with Windows software 12

13. The Participants 13

14. Why to Usability Test • What can usability tests achieve? • Identify usability problems • Ensure that a UI achieves its usability aims • Find superior designs of products • Improve the UI of a product • Make the product easier to use • Why should you usability test? • Your users are different from you. • Your users cannot always express usability problems. 14

15. When to Usability Test? • Before Product Launch: • A prototype is ready. • You anticipate some usability problems. • After Product Launch: • Customer complaints or requests for instruction on your toll-free number. • Customers are not purchasing the product. • Poor results from a usability test. • Reports of customer dissatisfaction from sales people. • A reporter or sales specialist finds your product inferior to competitors. • Negative results from consumer panel. • Marketing people in your organization find it hard to use. • A focus group indicates people dislike your product. 15

16. Why Not Test all Users? • The test requires time. • Approximate Time Required: 112 hours • 32 hours for designing test plan & material • 8 hours for designing the test environment • 4 hours for running a pilot test • 4 hours for revising test tasks/materials • 24 hours for running test/collecting data • 16 hours for summarizing data • 24 hours for documenting/presenting results 16

17. X Why don’t all companies Test? Usability testing is NOT expensive, but there are otherreasons : • They have already tested the products. • They don’t have a usability lab. • They don’t have time. • They don’t have contact information about their customers. • It is difficult to bring participants to the test. • There is no incentive for participants to come. 17

18. A Usability Lab or the User’s Environment? Advantages of a Usability Lab: Well managed No interruptions Convenient for recording Unobtrusive third-party observations Advantages of the User’s Environment: Authentic Economical Convenient for the user Where to Usability Test 18

19. Usability Lab 19

20. How to Usability Test • Required Steps for Testing: • Meet Preconditions • Make a Plan • Create Materials • Pilot the Test • Recruit the Participants • Test the Product • Summarize, Analyze, Report 20

21. Preconditions for Testing: Trial Product User Information: needs, aims, and related experience Planning the Test: Set a test target. Select a target group of users. Who best represents the product’s potential users? Select a specific task to be tested. Which task will give us the best insights into the products usability? Preconditions & Planning 21

22. LOGO Start 22

23. Why LOGO Start? • It’s the smallest product. • It’s simple to use: users don’t have to be experts. • A lot of customers use it. • It serves as a good model to be applied to more complex products. • The test was easy to implement. 23

24. Develop Test Script User Introduction Data Logging Sheet Post-test Questionnaire Test Tasks 24

25. Test Script (2) Test Day Introduction for Participants Thank them for coming/Briefly explain the product Clarify their (participants’) role Clarify moderator’s role Clarify the test’s purpose: “We are testing to IMPROVE not to prove.” 25

26. Tasks • Tasks are ... • Product Installation • Card Entries • Opening Slip Entries • Transaction Entiries • Reports 26

27. Posttest Questionnaire 27

28. Data Logging Form 28

29. Recruit pilot user Set up test environment Run pilot test Revise materials & environment Adjust timing We can test… Test documents Tasks Questionnaires Test room Observation room Data entry form Cameras Moderator Pilot Test 29

30. Decide whether you will do the work or hire a market research firm. Find people who match the user profile through phone calls, customer lists, or other databases. 3-10 users is ideal Inform of the test’s duration 1-2 hours per test with a 30 min. break between them. Consider the participants’ motivation. Should an incentive be offered? Tell them they have been selected to help with product improvement. Recruit Participants 30

31. The moderator Should Create a relaxed and comfortable atmosphere Ask questions to gain insight Ask post-test questions Inform participants they can discontinue Shouldn’t Be judgmental Lead or help Distract participants The observer takes notes During the Test 31

32. Summarize Count the number of errors per task Count the number of attempts per task Calculate the time used to complete each task Use descriptive statistics like averages Analyze Discuss immediate impressions as a group Go over the video Read user comments Read moderators comments and thoughts Discuss developers ideas: weaknesses/possible improvements Discuss the usability issues Summarize and Analyze 32

33. Report • List elements that worked • List problems: note frequency and severity • Be positive • Write reasons • Make suggestions for improvement 33

34. Participants’ Usability Problems 34

35. One of the Improvements 35

36. Conclusion • Time (but not much money) is required. • Companies already have necessary resources for the creating a testing lab. • It’s not necessary to hire a test manager from outside. • It’s not necessary to test all versions of a product. • Tasks should be selected carefully. • Participants should be selected carefully. • Consider possible scheduling and attendance issues of the participants. • Express the value of the participants’ role in product improvement. • Members of upper management should be in the observer group. • A short presentation should be prepared to present results of the test. 36

37. References • Peter P. Mitchel. A Step-by-Step Guide to Usability Test. iUniverse, Inc., New York Lincoln Shanghai, 2007. • Jerilyn Prescott , Matt Crichton, Usability testing: a quick, cheap, and effective method, Proceedings of the 27th annual ACM SIGUCCS conference on User services: Mile high expectations, p.176-179, November 1999, Denver, Colorado, United State. • Gitte Lindgaard , Jarinee Chattratichart, Usability testing: what have we overlooked?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA • Carol B. Mills, Usability testing in the real world, ACM SIGCHI Bulletin, p.212-215, 1986 • Mayhew, D. J. The Usability Engineering Lifecycle: A Practitioner's Handbook for User Interface Design, San Diego, 1999. 37