1 / 24

More and Better Test Ideas

More and Better Test Ideas. share one-liner test ideas. Rikard Edgren TIBCO Spotfire EuroSTAR 2009-12-03. Acknowledgements. This approach was “invented” and refined at my workplace, and at many other places as well A lot of good talking around test ideas: Kaner, Bach, Bolton

denzel
Download Presentation

More and Better Test Ideas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. More and Better Test Ideas share one-liner test ideas Rikard Edgren TIBCO Spotfire EuroSTAR 2009-12-03

  2. Acknowledgements • This approach was “invented” and refined at my workplace, and at many other places as well • A lot of good talking around test ideas:Kaner, Bach, Bolton • Many tips & tricks at many places • See paper for details and references

  3. Agenda • Test idea definition • Two examples • More test ideas • Better test ideas • Summary

  4. Test Idea Definitions • "a brief statement of something that should be tested.“ (Brian Marick) • “test condition: An item or event of a component or system that could be verified by one or more test cases, e.g. a function, transaction, feature, quality attribute, or structural element.”(ISTQB 1.1) • Test-Ideas List: “An enumerated list of ideas, often partially formed, that identify potentially useful tests to conduct” (Rational Unified Process)

  5. More Test Idea Definitions • ”A test idea is the thought that guides our creation of a test” (Cem Kaner) • “Test Idea: an idea for testing something” (James Bach) • maybe no definition is needed, you immediately understand what a test idea is.

  6. Example #1 • Spare time project from Canada • A tool for exploratory testers • Test specification from summer 2009 • Used for Beta release 0.2

  7. Example #1 • A combination of generic and detailed test ideas • 53 test ideas cover many important aspects • Possible to see what is missing • It is a good idea to combine several test ideas, and execute them together • Efficient to make the list while trying the software

  8. Scripted & Exploratory Testing • As approaches to software testing: • ST focus on control, precision • ET focus on learning and the freedom of the tester • As methods/activities/techniques: • ST designs and reviews tests in advance • ET design and execute simultaneously • Test ideas enable early feedback, add visibility, save time, and can act as a check list for ET

  9. Example #2 • Product development in Gothenburg, Sweden • Test specification from 2008 • Used for a 1.0 release • Good starting point for next project

  10. Example #2 • Test Ideas range from vague to very vague • Test Ideas can be converted to detailed test scripts, together with necessary details • Test Ideas can be executed as is, trusting the tester to design the tests and choose thoroughness

  11. More Test Ideas • Requirements aren’t enough • For each part and its interactions • What should be right? • What can go wrong? • What quality attributes matter? • What is important? • Be creative, and work hard • for each test idea, there are two more

  12. Test Idea Triggers • So many inputs, methods, heuristics... • requirements/specifications/prototypes/code • bugs/error catalogs/support/customers • technology/tools/systems/interactions/models • quality objectives/attributes • testing techniques • test idea lists • people • your subjectivity, knowledge, experience...

  13. My Favorites #1 • Quality attributes • Capability • Reliability • Usability • Security • Scalability • Performance • Installability • Compatibility • Supportability • Testability • Maintainability • Portability • Localizability • Accessibility • Interoperability

  14. My Favorites #2 • Delete/Remove/Empty/Null • Read the manual • Look at all deliverables • Use the same data/document a very long ”time” • ”Real” usage of the product

  15. Better Test Ideas • Powerful • Yield significant results • Credible • Likely • Easy to evaluate • Useful for troubleshooting • Informative • Appropriately complex • Giving insightful information • Easy to understand* • Fast to execute* * not in Kaner’s What is a good test case

  16. Better Test Ideas • Test ideas get better through collaboration • ...with people that care, and understand what’s important • Try to get feedback from many sources • Discard test ideas that seem weaker • You will get even better test ideas as you learn more about the product

  17. Test Ideas in Practice • Many different types • One-liners are fast to read • Easy to update • Vague to leave freedom • Useful now-then, me-otherpeople • Adjust granularity for better readability

  18. Find Five Faults ...but there are other things than these faults that are more important...

  19. Limitations • Test executors might need a lot of testing and product knowledge • Might be difficult to evaluate results anyway • Can be difficult to get good feedback • Generic test ideas might obscure the new context

  20. Summary • Many different test ideas is a good thing • More people will give better perspectives • Test ideas are fast to read, write and update • Test ideas could be the glue you need for using test scripts and an exploratory testing approach • Documented test ideas is a simple idea, you need to use it to understand the value

  21. Questions • ??? www.thetesteye.com redgren@tibco.com

More Related