1 / 28

High Volume Automated Testing for Software Components

High Volume Automated Testing for Software Components. Harry Robinson Doug Szabo Microsoft. High Volume Test Automation.

preston
Download Presentation

High Volume Automated Testing for Software Components

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. High Volume Automated Testing for Software Components Harry Robinson Doug Szabo Microsoft

  2. High Volume Test Automation The essence of HVTA techniques is automated execution and evaluation of large numbers of tests, for the purpose of exposing functional errors that are otherwise hard to find. Experiments with High Volume Test Automation, 2004 McGee & Kaner

  3. Subtle Bugs Only weak bugs have a logic to them and are amenable to exposure by strictly logical means. Subtle bugs have no definable pattern - they are wild cards. Software Testing Techniques 2nd Edition, 1990 Beizer

  4. Bounded Exhaustive Testing Bounded exhaustive testing (BET) is a verification technique in which software is automatically tested for all valid inputs up to specified size bounds. Software assurance by bounded exhaustive testing, 2005 Coppitet al

  5. Typical Biases • Developers – “happy paths” • Testers – “a nose for bugs”

  6. CodeProject: Sorting Algorithms in C#

  7. The Sorting Algorithms

  8. Popular …

  9. Recommended …

  10. … and it looks cool!

  11. … but dark clouds appear on the horizon

  12. . . .

  13. Reasonable Unit Tests

  14. … Even Some Randomized Unit Tests

  15. Pre-Oracled Data shuffle compare sort

  16. The Loop

  17. Shearsort demo #1

  18. Shearsort demo #2

  19. Shearsort bug

  20. Shearsortbug - fixed

  21. Heapsort bug

  22. Heapsort bug - fixed

  23. OddEvenTransport bug

  24. OddEvenTransport bug- fixed

  25. Distribution of Buggy Array Sizes

  26. Distribution of Buggy Array Sizes Bug is not detected at array size 1000

  27. Number of Random Arrays to Detect a Bug

  28. GeneratedUnitTests

More Related