1 / 25

Human testing of mobile apps

Human testing of mobile apps. For Vistacon 2013 conference 25 April 2013. Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution- ShareAlike 3.0 Unported License. JULIAN HARTY.

luka
Download Presentation

Human testing of mobile apps

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Human testing of mobile apps For Vistacon 2013 conference 25 April 2013 Creative Commons License How to design your mobile apps by Julian Harty is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License. JULIAN HARTY http://creativecommons.org/licenses/by-sa/3.0/deed.en_US Contact me: julianharty@gmail.com Rev: 29 April 2013

  2. Drive for efficiency automation

  3. State of affairs • The industry is driving towards automated tests • ‘Manual testing’ is downgraded • Agile projects need automated tests to provide fast & frequent feedback • Development environments for Mobile Apps (SDKs) include Test Automation frameworks

  4. Automated tests • Unit Tests • Continuous Builds • UI Test Automation • Scale out to run tests on multiple devices • Virtual devices, remote devices, local devices • Complex, sometimes fragile, ecosystems • Challenges: device security models, limitations in frameworks, connectivity, immature domain, lack of support for multi-modal interfaces, etc.

  5. Human-centric testing People need People too

  6. Beyond the reach of automated tests • Test frameworks focus on code, API, or GUI, • Not: • Movement: acceleration, rotation, location • Tactile, Haptic, audio Inputs or Outputs • Aesthetics: look & feel, transitions • User Experience (UX) • Highly dynamic updates: video, audio output • Complex inputs: Gestures, multi-touch, shaking • Light: e.g. camera as an input device • So how do we test these?

  7. QOX • Quality Of Experience (QOX) • From the standpoint of a user’s holistic experience of using the mobile app. • Essentially subjective UX Research, Evaluation, and Testing: strategies, tips and tools Mark J. Williams http://en.wikipedia.org/wiki/Quality_of_experience

  8. I’ve got feelings too “An inconsistency between the product and the document might be surprising, frustrating, or annoying to that person, but what really matters is inconsistency between the product and the desire.” • 6 hats of software testing • The ‘red’ hat – emotions, intuitions, feelings I've Got a Feeling: Emotions in Testing http://www.stickyminds.com/s.asp?F=S17922_COL_2 Here's an article I wrote a few years ago on the six hats of software testing http://archive.newsweaver.com/qualtech/newsweaver.ie/qualtech/e_article000666359.html and if you really have lots of time on your hands, here's a video of a talk I gave on the same topic at StarWest 2008 http://www.stickyminds.com/Media/Video/Detail.aspx?WebPage=117

  9. So… What do we need to do?

  10. Fast, efficient interactive testing • Bugfests • Move and rotate the phone • Be mobile Bugfest: A group of people get together to test in parallel and in common. Developers on-hand for direct feedback.

  11. Lunch outtesting a video app • Around 2 hours • Around 10 testers • Test the app while going out for lunch • Examples of what the testers did • Tourist trip on the top of a double-decker bus • London Underground • Out for a walk with sandwiches

  12. Establish strong foundations • Learn: • The platforms, the norms, what users expect • How the platform was designed to be used • The application lifecycle • The devices • Common, and unusual problems

  13. Insightful testing • RAPID software • Heuristics • Mnemonics to guide the testing • Session-Based Test Management techniques • Charter • Time-boxed • Simple, lightweight test reporting • Debrief http://moolya.com/blog/2012/04/18/test-mobile-applications-with-cop-who-flung-gun/

  14. Extended testing

  15. Extended testing • Remote devices • Remote testing

  16. Remote testers Credit: "Ken Banks, kiwanja.net" http://www.kiwanja.net/gallery/texting/kiwanja_kenya_texting_17.jpg http://qz.com/49916/indian-smartphone-users-are-gobbling-up-mobile-data/ http://www.imedicalapps.com/2012/09/researchers-smartphone-spirometer-mobile-app/

  17. Challenges of extended testing • Loss of information • Sensory deprivation • Missing or incomplete data • Loss of fidelity • Lags, delays • Loss of control How do we test rich interactive apps unless you have the device in your hand? “In theory, theory and practice are the same. A device in your hand is worth 2 in the cloud, and 100 virtual devices. In practice, they are not.”

  18. More context

  19. More context • Additional considerations • Cultural norms & differences • Generations • What’s important to them? • How can we empathize? A South Korean woman using her mobile phone to buy a product at the virtual retail shop in Seoul. www.guardian.co.uk Culture? Culture? Mobile Apps & Platforms Culture? International GSMA REVEALS FEARS OVER MOBILE PRIVACY ARE HOLDING BACK THE GROWTH OF MOBILE APPS IN LATIN AMERICA http://www.gsma.com/newsroom/gsma-reveals

  20. Examples of what can go wrong • “What this means is that unlike the iPhone, which is most certainly the closest competitor on the market to this phone, the Storm's UI is not custom built for touch navigation -- touch navigation is added after the fact. Things which flow naturally on an iPhone -- flicking through lists, scrolling for a contact, moving around in a webpage or looking through photos -- feel inelegant and uncomfortable on the Storm. There's no inertia to movement, no assurance that your finger is the lynchpin to control of the device. The screen is sensitive enough, surely, but how its software reacts to those touches makes all the difference, and here the feeling is that you're never completely in charge of the phone.” http://www.engadget.com/2008/11/19/blackberry-storm-review/ Image: http://worldwide.blackberry.com/gettingstarted/storm-9500.png

  21. Practical outcomes ! XOR[*] [*] Not Exclusive OR

  22. Combining strengths • Automated Tests • Fast feedback, technology and developer centric • Interactive Tests • Exploratory, rich interactions and dynamics • Remote testing e.g. in other countries • Combined • Augmented, Cyborg testing e.g. automated preparation, data collection, etc. Learn to delegate & learn when to take control

  23. Some further readings Testing and Test Automation for Mobile Apps Julian Harty Summer 2013 CRC Press Marketing Development Testing(2009) Testing(2013) http://www.wipconnector.com/download/GuideToTheParallelUniverse_3rdEdition.pdf http://www.enough.de/fileadmin/uploads/dev_guide_pdfs/Guide_12thEdition_WEB.pdf

  24. Q & A ?

  25. If you want the draft book • Draft book available for review & feedback • Email me julianharty@gmail.com • Agree not to share or distribute • Comment to get the next available draft

More Related