1 / 52

Mobile Usability Testing

Lecture :: „Mobile Usability Testing“. Mobile Usability Testing. Topics Motivation & Definition Mobile Devices Challenges in Mobile Usability Testing Methods & Approaches Heuristics & Guidelines Examples & Ideas. Inst. f. Softwaretechnik und Interaktive Systeme

opa
Download Presentation

Mobile Usability Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture :: „Mobile Usability Testing“ Mobile Usability Testing • Topics • Motivation & Definition • Mobile Devices • Challenges in Mobile Usability Testing • Methods & Approaches • Heuristics & Guidelines • Examples & Ideas Inst. f. Softwaretechnik und Interaktive Systeme qse.ifs.tuwien.ac.at, thurnher@qse.ifs.tuwien.ac.at, christian.fruehwirth@qse.ifs.tuwien.ac.at

  2. Goals ::What do we want to achieve in this lecture? Mobile Usability Testing • What we want: • Enable you to set up a Mobile Usability Test based on Quantitative as well as Qualitative Methods. • Give you Inspirations & Ideas about what can be done to make mobile applications more usable. *vgl.: Schilit and Theimer (1994) 2

  3. Methods ::How do we want to achieve our goals? • How we want to do it: • Creating an awareness for the challenges of mobile-Usability • Compare common Usability Testing Methods • Overview about possible Heuristics & Guidelines for Mobile Usability Testing Mobile Usability Testing *vgl.: Schilit and Theimer (1994) 3

  4. Definition ::„Mobile Usability Testing“ I. Motivation & Definition • Considerations for Mobile Testing: • Results are strongly influenced by surrounding environment • Results are influenced by devices used for testing • Collected Data will be „richer“ (Gesture, Voice, …) 4

  5. Mobile Usability Testing I. Motivation

  6. Costs of late Usability research I. Motivation & Definition For every dollar spent acquiring a customer you will spend $100 dollars reacquiring them after they leave because of poor usability or bad customer service. (*) *vgl.: MauroNewMedia (2002) 6

  7. Usability Engineering in the Software Development Process I. Motivation & Definition *vgl.: INTERACT 2001 Workshop, Jan Gulliksen, Inger Boivie) 7

  8. Mobile Usability Testing II. MobileDevices

  9. Mobile Devices :: Definition Are used “on -the-run and for activities that may last only a few seconds or are highly context dependent” (*) II. Mobile Devices *vgl.: (Vetere et al., 2003, p.1) 9

  10. Mobile Devices :: Realistic View 1 • „Mobile Devices: One Generation From Useful“(*) • Tighter Integration needed: • Devices do not work together well with each other. Synchronization with other Applications • Different Features packed into one device do notact as one entity  PDAs with GSM modules • Ad-hoc Networking is still far from realisation Network coverage & Roaming (WLAN, UMTS, GPRS, GSM) Industry Standards (Bluetooth compatibility, vulnerability) II. Mobile Devices *vgl.: (Jakob Nielsen 2003, Alertbox , Aug. 18th.) 10

  11. Mobile Devices :: Realistic View 2 • Design / Interface Problems still to be solved: • Deck-of-Card Size(*) limits the Screen Size Higher Screen-Resolutions, better viewing angle. • One Dimensional Interfaces (Scroll wheels) are not suitablefor 2D – Screens. • Text-input is still a great Problem.Small Devices  small Buttons.  new Button Alignments? II. Mobile Devices *vgl.: (Jakob Nielsen 2003, Alertbox , Aug. 18th.) 11

  12. Mobile Devices :: Realistic View 3 • Fundamental Problems: • Quality of Service of local Network Providers.Things that „could be done“ just can‘t because of local Service-Limitations or lack of network coverage. (Broadband, UMTS, etc.) • Online Services must specialize for Mobile useMuch shorter Articles, more use of XML, simplyfied Navigation • Reconsider the way email is used  not just forward every mail to the PDA (Attachments, Executeables) II. Mobile Devices 12

  13. Mobile Usability Testing III. Challenges

  14. Challenges ::of Mobile Usability Testing (*) • 1. Device Proliferation • Handling many different Devices, Rendering Methods • 2. Application Modality • Handling simultaneous voice / Data User interactions • 3. User Mobility • Users are likely to be distracted during use • 4. Data Collection • Recording eye-movement and video taping will not work everywhere III. Challenges *vgl.: http://www.littlespringsdesign.com/analysis/utest.html 14

  15. Challenges :: 1 : Device Proliferation (*) • Devices can be • As small as possible, optimized for voice communication • Quite large, optimized for data display • Optimized for gaming • Optimized for multimedia III. Challenges • Applications are perceived differently • Reading News-Bulletin on a point-matrix phone display  content is forgotten 3 hours later. • Reading News-Bulletin on a java-enabled 19“ CRT Monitor Will be stored in long-term memory *vgl.: http://www.littlespringsdesign.com/analysis/utest.html 15

  16. Challenges :: 2 : Application Modality(*) • Mobile Applications often combine Graphic / Data and Voice Elements. • Difficult to test in an early stage of development(Software isn‘t fully functional / not yet written.) • Test must be able to provide simultaneous experiences III. Challenges *vgl.: http://www.littlespringsdesign.com/analysis/utest.html 16

  17. Challenges :: 3 : User Mobility • Mobile Users are very likely to be distracted. • Natural Environment is not always manageable/affordable to be simulated in a lab • Distractions and „Normal anomalies“ (Waiter interrupting you in a restaurant to take your order) have to be part of the test III. Challenges 17

  18. Challenges :: 4: Data Collection 1 • Acquiring Data outside the Lab • Eye tracking will hardly work on tiny screens and under mobile conditions. • Recording tools interfere with the users‘ interaction with the device.(Cameras mounted on a cell phone make the user hold it in an unnatural position) III. Challenges “It is cold and snowing and you do not know from where your bus leaves in 5 minutes. You pick up your WAP phone to check: The mobile user runs to catch her bus, after her run three researchers with cameras and microphones..(*)“ *(vgl.: Per-Ola Rasmussen ExarbII 2003) 18

  19. Challenges :: 4: Data Collection 2 • Mobile Users interact not just with the screen and keyboard. • Test needs to record, gestures, face-expressions, voice, body-language, etc. •  Much richer Data (People leaning left and right while playing a Formula-1 racing Game on their Java-enabled Phone.) • Record what is the user doing, what is he/she NOT doing. III. Challenges 19

  20. Mobile Usability Testing IV. Methods & Approaches

  21. Methods :: Approaches to Mobile Usability Testing 1 • The common Methods IV. Methods & Approaches *(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000) 21

  22. Methods :: Approaches to Mobile Usability Testing 2 • The common Methods IV. Methods & Approaches *(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000) 22

  23. Methods :: Approaches to Mobile Usability Testing 3 • The common Methods IV. Methods & Approaches *(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000) 23

  24. Methods :: Approaches to Mobile Usability Testing 4 • The common Methods IV. Methods & Approaches *(vgl.: Tomas Lindroth, Stefan Nilsson & Per-Ola Rasmussen, ExarbII – HT2000) 24

  25. Methods :: Lab vs. natural environment The larger the number of factors that is under control in a test, the more scientific rigour is emphasized. The more natural like the test setting is, the more relevant and applicable the results will be. IV. Methods & Approaches *(vgl.: Mason 1988, Järvinen, 1999) 25

  26. Methods :: Approaches to Mobile Usability Testing 5 • The Ideal Test • Natural situation / environment • Application is fully functional • All possible forms of devices are being tested • Users are free to do what they would normally doUsers don't feel „tested“ • Tester can record every audio / visual / voice / movement / screen datafrom the user without affecting the users behavior. • Tester sees the Application the way the users see it. IV. Methods & Approaches 26

  27. Methods :: General Mobile Usability Testing Process 1 • Preparation • Define goals, methods and tasks/scenarios for the test. Introduction, Warm-up • Introduce the test to the user • Start with easier tasks, give time for a short warm-up phase. Testing • Perform the actual test with as less interaction between user and tester as possible IV. Methods & Approaches 27

  28. Methods :: General Mobile Usability Testing Process 2 Test situation • Give the user time to get out of the test-situation. • Then start reviewing his/her opinions, impressions and suggestions. • Make sure to discuss special occurrences that may have happened during the test with the user. IV. Methods & Approaches 28

  29. Remember ::Challenges to deal with (*) • 1. Device Proliferation • 2. Application Modality • 3. User Mobility • 4. Data Collection IV. Methods & Approaches *vgl.: http://www.littlespringsdesign.com/analysis/utest.html 29

  30. Methods :: Handling User Mobility 1 • For informal, problem identifying tests • Ask participants to use the application maybe over lunch • Offering a compensation helps „motivating“ the participant • Don‘t forget to have them sign an informed consent statement • Interruptions (waiter, etc.) are welcome Watch what happens when the users resumes the task and see what difficulties occur. IV. Methods & Approaches 30

  31. Methods :: 3 : Handling User Mobility 2 • For formal, statistically precise tests • Don‘t try to introduce distractions into the test unless you are testing with a greater number of participants • Referring to Nielson, the marginal benefit will decrease if you are testing with more then 10 UsersThesis and formula is questioned from many researchers. IV. Methods & Approaches 31

  32. Methods :: 3 : Simulate the natural environment? 1 • Chances(*): • Reproducible conditions • Easier / more complete documentation • Use of more sophisticated tools Risks • Unusual environment for the user • Restrictions due to simulation • Non recording of the original work surrounding field (office atmosphere, disturbances, etc.) IV. Methods & Approaches *(vgl.: akziv. Requirements from users point of view. 2004) 32

  33. Interactive Example • Which environmental factors are reproduceable in a laboratory surrounding. Environmental Conditions BT 041118.xls IV. Methods & Approaches 33

  34. Methods :: 3 : Simulate the natural environment? 2 • „Mobile Devices are build to be mobile so take them out into the field“ (*) • Take the lab to the user, not the user to the lab. IV. Methods & Approaches *(vgl.: akziv, „wearability“. 2004 34

  35. Methods :: 3 : Lab Test vs. Field Test(*) 1 • Example Application: Using SMS Service on a PDA while walking • In the lab: on a treadmill • In the field: on a pedestrian street IV. Methods & Approaches *(vgl.: Jesper Kjeldskov, Aalborg University Denmark) 35

  36. Methods :: 3 : Lab Test vs. Field Test(*) 2 IV. Methods & Approaches *(vgl.: Jesper Kjeldskov, Aalborg University Denmark) 36

  37. Methods :: 3 : Lab Test vs. Field Test(*) 3 Usability problems identified by the test subjects • Numbers are basically equal • Notice that while sitting on a desk the cosmetic problemsidentified by the users were far more! IV. Methods & Approaches *(vgl.: Jesper Kjeldskov, Aalborg University Denmark) 37

  38. Methods :: 3 : Lab Test vs. Field Test(*) 4 Are Lab tests superior? • Consider Cost / benefit of different techniques and settingsTime and effort per problem found • Can you afford NOT to find a problem? • Costs of missing Usability: • lost of repurchases • increased calls at helpdesk • lost of repurchases • lost of brand reputation • necessary redesign in late state or next version • law suits • …… IV. Methods & Approaches *(vgl.: Jesper Kjeldskov, Aalborg University Denmark) 38

  39. Methods :: 4 : Handling Data Collection 1 • Mobile devices are extremely personal. Users may pick them up, gesture, or lean back with them.(*) • Record Device Screen + Users Face at the same time • Use wireless tracking & recording technology(WLAN, Bluetooth, small radio cameras, etc.) •  Users should not be handicapped by the testing equipment IV. Methods & Approaches *(vgl.: Little Springs Inc. 2004) 39

  40. Methods :: 4 : Handling Data Collection 2 • Recording a mobile phone‘s screen and the users face at the same time with two cameras mounted on the phone(*) IV. Methods & Approaches *(vgl.: Little Springs Inc. 2004) 40

  41. Methods :: 4 : Handling Data Collection 3 • Recording „Soft Information“ • Define methods to integrate „soft-information“E. g.: Users starts shaking the Phone to make it start the application faster; user‘s thumb tends to cover up parts of the device‘s display) • Many qualitative information can be quantized. • Measuring heart-beats / second to determine the stress-level • Recording subconscious hand and leg movements. • Record number of extra-applicational interactions(user answered 2 phone calls and asked his colleague for help during the use of the application for 5. min.) IV. Methods & Approaches *(vgl.: Little Springs Inc. 2004) 41

  42. Mobile Usability Testing V. Possible Heuristics & Guidelines

  43. Heuristics :: General Guidelines for mobile Applications 1 • Highly functional design • Don‘t use fancy designs if they don‘t bring a real benefit for the User • Consistent usage of icons / buttons / names and labels • Consider the users mental-models when you introduce new functions or name buttons. • Integrated content navigation • Help the user tracking it‘s way through the information, always provide a clear exit – point. • Consider Shortcuts V. Heuristics & Guidelines *(vgl.: Little Springs Inc. 2004) 43

  44. Heuristics :: GeneralGuidelines for mobile Applications 2 • Reduced HCI interactions • Especially when done on mobile devices interactions with the user are often difficult and time-consuming (text-input on a mobile phone) • Reduce Interactions by any means possible (Location based services, Heuristics, default-values, etc.) • Offer intelligent search-functions • Assist the user in finding the information, as any unnecessary interaction makes the applications less usable for the user. V. Heuristics & Guidelines *(vgl.: Little Springs Inc. 2004) 44

  45. Heuristics :: Mobile Games 1 • For Mobile Games the Rules are • a little different • Navigation Consistency? • The User should not feel like using his/her phone, he/she should experience the Game World V. Heuristics & Guidelines *(vgl.: NOKIASeries 60 Developer Platform 2.0: Usability Guidelines For J2ME™ Games 45

  46. Heuristics :: Mobile Games 2 • Game experience vs. Social acceptable behavior • Sound, Light and Vibration enhance the users Game experience •  Typically, games are played in locations where it is not suitable or socially acceptable to have the sound on. • During Mobile Usab. Testing consider there are usually other people close by when the user plays the game. V. Heuristics & Guidelines *(vgl.: NOKIASeries 60 Developer Platform 2.0: Usability Guidelines For J2ME™ Games 46

  47. Mobile Usability Testing VI. Examples & Ideas

  48. Text Interface :: Projecting the image • A standard-sized Query-Keyboard is projectedby laser on any given surface. • The Users input is recognised by a small camera in the cigarette-pack sized device. • Note: Being announced in 1999 the product is still under development and may never reach market maturity. (2004)(*) • www.virtualdevices.net VI. Examples & Ideas *(vgl.: http://www.ibizpda.com) 48

  49. Text Interface :: Breaking the Qwerty Paradigm • Standard-sized keys aligned for one-handed use. • Can be used under mobile conditions (doesn't need chair + desk environment as similar fold-up keyboards for PDAs) VI. Examples & Ideas *(vgl.: www.frogpad.com/) 49

  50. Remote Controll :: Force Push • Operate household devices with agesture of your fingertip. • IR-Led points at Device to be controlled • Touch & Acceleration Sensors combinedwith Software recognise gestures and execute command VI. Examples & Ideas *(vgl.: Koji Tsukada, mobiquitous.com/pub/apchi2002-ubi-finger.pdf/) 50

More Related