110 likes | 240 Views
Exploratory Testing. What it IS. A rapid cycle of Design a test Execute it Observe results. Simultaneous learning, test design and test execution Unscripted Targeted Repeatable Relevant. Until we've characterized the capabilities and limitations of the system with respect to a charter.
E N D
What it IS • A rapid cycle of • Design a test • Execute it • Observe results • Simultaneous learning, test design and test execution • Unscripted • Targeted • Repeatable • Relevant Until we've characterized the capabilities and limitations of the system with respect to a charter
What it’s NOT • Not Functional testing • Tests are not scripted ahead of time • Results are not necessarily known • Not Random testing • Requires a human mind to execute testing and/or interpret results • More focused/directed than random testing • Not “ad hoc” testing • There is a Charter to structure the area of exploration • Tester understands how the user interacts with the application, and knows the historical weak points of the application
Charter • A mission statement consisting of two or 3 sentences to guide your testing for the session. Ex: Analyze the X function. Make note of any risks, areas of instability or latency. • Manages scope • Create charter prior to testing. • Explore: story, feature, heuristics, system, testing tour • With: resources, constraints, features, setup • To discover: information
Session Based Test Management • Time frame each session ~90 min • What’s the best test I can perform right now? • Use Notebook or some other recording method
Why does it Work? • Testers use results of previous results to guide their future testing on the fly • They do not have to complete a series of scripted tests before focusing in or moving on to explore a more target rich environment • This accelerates bug detection when used intelligently
The Theory • Discovery helps you choose relevant things to do among the infinite number of possibilities • The variations possible are “learned” as other tests are run • You run tests you didn’t think of initially – can’t anticipate everything! • “Sampling” vs complete coverage of conditions where complete coverage is not practical • Tests drawn from using a testing technique the team normally doesn’t apply • Tests are not written ahead of time – fast, flexible • You log as you go – requires practice, new habit • Does knowing more about the intended use help you do exploratory testing better? More efficiently?
When are you DONE ? • Testing does not reveal NEW useful information. • Experienced tester feels comfortable. • Critical risk features have had time to go through exhaustive testing.
More Exploratory Testing Tidbits • Tested = checked + explored • Comprehensive, repeatable regression tests are nice. But not sufficient. • Requirements are missed • User’s do unexpected things • Discover things you can vary • Use state/transition models • Use data models • Use deployment diagrams • Learn things useful for process AND product • Don’t go off the deep end • Explore early, explore often
Appendix B: Testing Heuristics links • Elisabeth Hendrikson Test obsessed - http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf • James Bach Satisfice - http://www.satisfice.com/tools/htsm.pdf