Software Testing Techniques

1. Software Testing Techniques Dan Berger dberger@cs.ucr.edu Peter Fr�hlich phf@cs.ucr.edu

2. Outline What is Testing? Why Test? How To Test The Tests? Code Coverage (Peter) How To Decide What To Test? Automation Tools Writing Test Cases

3. What Is Testing? In this context � confirming correct behavior of a piece of software.

4. Why Test? We know what we intended the code to do � we need to confirm that it does what we intended.

5. How To Test Our Tests? In other words: how can we be sure our tests are: testing the right (relevant) �stuff�? testing �most� of the code and not just repeatedly testing the same code?

6. Code Coverage Take it away, Peter�

7. How To Decide What To Test?

8. How To Decide (II) The trick is drawing a line across that triangle. Draw it too low and you have too many classes to test � and the tests aren�t meaningful. Draw it too high and a test failure leads you on a wild goose chase� The bad news is � deciding where to draw that line is an art, not a science.

9. Automation Tools How many times have you written code and then sat in front of the keyboard running through test cases? How many times have you actually thought about test cases? The good news is that there is a better way. And it�s never too late for a new beginning�

10. Automation To The Rescue A few key tools: make make running tests a part of your build scripting if you can automate it, automate it expect a powerful tool

11. Make Add a test rule to your makefile � it should run all the tests you�ve written and log their results. If they all pass, you�re done � if not, figure out why not.

12. Scripting Imagine you have a bunch of executables that test various parts of your program. We�ll talk about how they might be built shortly. Imagine you�ve written a shell script to run them, one at a time. How do you know when one fails?

13. Scripting (II) You can check the exit code of a program running from a shell script: if [ $? �ne 0 ]; then � else � fi if the executable exits abnormally, the then clause executes.

14. Expect Written by Don Libes � an extension to Tcl What, you�ve never heard of tcl? That�s OK � there�s a python version as well. If you�ve never heard of python, you should have attended our python seminar�

15. Expect (II) It�s be installed on the lab machines Basic commands: spawn run�s a new command under script control. send provides input to the command expect matches a pattern in the command output

16. Expect (Example) #!/usr/bin/expect set timeout 5 spawn ssh �x [lindex $argv 0] �ls /� expect { timout {puts �timeout�} eof {exit 0} �continue connecting (yes/no)? � {send �yes� } } expect { eof {exit 0} }

17. Python Expect Pexpect � http://pexpect.sourceforge.net Good News: Provides *most* of expect�s functionality from within python. Bad News: it�s not installed on the lab machines (yet?).

18. Writing Test Cases Write a simple executable that instantiates a class, and �drives� that instance.

19. Test Cases (Cont.) Imagine you have a linked list class� If you wrote your own linked list class � and you�re working in C++ - you should have been in our STL seminar. What should your test program test?

20. Parting Words Be a skeptic � you, as the developer, are the only one who can crystal-box test your code. Don�t test the easy cases � test the ones you�re not sure about. Don�t waste time testing the same thing more than once � break input into �equivalence classes�