A Walk on the Web Side

1. A Walk on the Web Side Differences and Similarities Between Web and non-Web Application Testing WISQA Special Presentation June 11, 2003

2. Overview • Functionality and Reliability testing: making sure a website works as required • Compatibility: making sure a website works as required for the users’ many operating systems and browsers • Usability: making sure a website works to meet users’ goals with effectiveness, efficiency, and satisfaction

3. Functionality and Reliability Testing

4. Functionality & Reliability: The Usual Suspects • Forms and form validation • Error messages • Proofreading the content • Checking calculations • Verifying that new entries are saved correctly • Verifying that changes are saved correctly • Verifying that deletions are really deleted • Find the weaknesses in system under test • Regression testing

5. Functionality & Reliability: Web-wise • Update releases: when live, users see instantly • Navigation: users have choices outside of what the site itself offers; page orders aren’t strict • Browser settings: users can control many things about their browser… how many combinations will you test? how many directly affect the site? • Cookies: should the site work without them? • Search: what can this site’s search functions do? • Are there web-unique functions for the site… like uploading or downloading files? (snapfish.com) or sending email? (cnn.com)

6. Functionality & Reliability: Complications of eCommerce • Business rules vary from online seller to online seller, so understanding them quickly is vital • Is registration (saving of name, address, and possibly credit card number) required before checkout? Can it be done during or even after checkout? • Are shipping and billing addresses saved separately or together? Can they be edited, or must they be deleted and re-entered? • What are the rules for sales tax calculation? • What parts of the site must be secure?

7. Functionality & ReliabilityPlaying with Test Cases www.rei.com Let’s buy a roof-mounted bike rack for the car. What test cases do we see along the way? • Navigation • Search • Registration • Checkout • Site-specific features

8. Functionality & ReliabilityHandy Tools We Use • Many tools for automated testing of web sites exist (see www.testingfaqs.org) • Defect tracker: TeamTrack by TeamShare • SnagIt by TechSmith, super screen capture • Realistic Random Data generator • Microsoft Excel • Paper techniques: • Page flows – helps plan workflow cases • Page hierarchy diagrams – helps plan navigation cases • Page mockups – identifies page elements requiring cases

9. Compatibility Testing

10. Compatibility Testing • It’s a browser jungle out there: • Internet Explorer 6, 5.5, 5, 4… • Netscape Navigator 7, 6.2, 6.1, 6, 4.79, 4.78… • Opera • Mozilla • Safari • AOL • More! www.download.com offers 192 other browsers! • Don’t forget the operating systems: Windows (95, 98, 2000, ME, XP, NT), Mac (OS 9, OS X), Linux… How could you possibly test them all?

11. Compatibility TestingMaking Good Choices • First step is to help the customers choose which OS/browser combinations to support • Do they have web logs for a current site? • Do they have a specialized customer base? • Get national statistics, watch the news • Consider support for more users vs. technology: if you must use a technology (for example, Cascading Style Sheets), who might this disenfranchise? (And does the customer care?)

12. Compatibility TestingCreating a Test Lab • Create a plan • What browsers and operating systems do you need? • What limitations are there (e.g., can’t install multiple IE versions on one machine)? • What hardware do you need? (machines, browsers, hubs, KVM switches for monitors & keyboards) • What software do you need? (browsers, screen capture, instant messenger, etc.) • Try for a modem and phone line • Go scavenging • Old PCs useless to company employees are invaluable to your lab

13. Compatibility TestingMuppet Labs • “Muppet Labs” lives in a corner of our building • 9 PCs • 3 Macs • 9 Operating Systems • 21 Browsers • Lab area is labeled with useful information:

14. Usability Testing

15. First, a few definitions… • User Interface - The language by which a user and system communicate with each other. • Usability - The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use. (ISO 9241-11 - Ergonomic requirements for office work with visual display terminals (VDTs) - guidance on usability)

16. More definitions… • User Centered Design (UCD) - any of a wide variety of methods and techniques for designing usable products. Key features: • Understand users - who they are & what they need • Understand the context of use • Involve users in the (iterative) design process • Evaluate designs against requirements (ISO 13407: Human-centered Design Processes for Interactive Systems)

17. And more definitions… • Usability Evaluation - any of a class of methods used to assess/evaluate/review how usable a particular product is. • Expert evaluation • Competitive evaluation • Task analysis • Cognitive walkthrough • User test • etc • User Test - Facilitated evaluation of the usability of a product by asking real users to perform real tasks with real products in real contexts. • Or as close as you can get to “real”

18. Web vs. Desktop • Ease-of-learning vs. ease-of-use • Much more holistic user experience • Nobody reads on the web • “use, then buy” vs. “buy, then use”

19. What to Use When • Competitive Evaluation • Product definition • Planning for redesign • Helps identify areas of concern, strategic advantages, etc. • Define what you want to build • Task Analysis, et. al • Analysis & design • Helps ensure you’re solving the problem the right way • Define how you want to build it

20. What to Use When (cont) • Expert Evaluation • Early (and often) during initial design phases • As soon as you have anything to evaluate • Very large bang for the buck - find most of big gotchas early • User Testing • Not-quite-as-early (but still early) and often • Certainly before launch • You’re going to user test it anyway, might as well do it before launch • A bit harder, but only way to get at some issues

21. Lo-fi vs. Hi-fi • Don’t wait until you have a working system (or even prototype) to test • You can test a piece of paper • Sometimes, low-fidelity is better to test (fewer misconceptions) • Fixing mistakes during construction costs ten times as much as fixing during design • You wouldn’t dream of not testing what you build… why would you ever NOT test what you design?

22. Let’s Practice! • www.intellicast.com • What’s the forecast for Buffalo, NY be tomorrow? • Find a current radar image for Honolulu • www.usatoday.com • Find the box score for the Milwaukee Brewers baseball game yesterday • Find some current news about the NHL playoffs

23. Recommended Reading • Basics of web testing: The Web Testing Handbook by Steven Splaine and Stefan P. Jaskiel • For software with a web front end: Testing Applications on the Web by Hung Q. Nguyen • Usability: Don't Make Me Think: A Common Sense Approach to Web Usability by Steve Krug • Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests by Jeffrey Rubin • See www.stickyminds.com for more books and articles