Automated Unit Testing

1. Automated Unit Testing Getting Started The 2008 DC PHP ConferenceJune 2nd, 2008

2. Hello! • Mike Lively • Lead DeveloperSelling Source, Inc. • PHPUnit Contributor • PHPUnit Database Extension • A frequent inhabitant of #phpc • http://digitalsandwich.com

3. Automated Unit Testing • What is unit testing? • Why unit test? • What do I test? • When do I test? • How do I test it?

4. Automated Unit Testing • What is unit testing? • Why unit test? • What do I test? • When do I test? • How do I test it?

5. What is Unit Testing • From Wikipedia: Unit testing is a procedure used to validate that individual units of source code are working properly. • Tests should test as little as possible. • Each test should validate a single unit. • Unit testing is just a single aspect of quality assurance. • Unit Testing • Functional Testing • System Tests • Acceptance Tests • Code Review • Etc.

6. Automated Unit Testing • What is unit testing? • Why unit test? • What do I test? • When do I test? • How do I test it?

7. Why should I unit test? • PHP is being used more and more for business critical applications. • There are two periods in which bugs will be found: during development and after development. When would you rather find your bugs? • Regression testing - make it safer to change code.

8. Why should I unit test? • Makes integration testing and functional testing less stressful. • Provides built in examples of code use. • Can be used effectively as a design validation tool. • Code that is hard to test is often hard to reuse. • Tests will expose difficult to use APIs.

9. Automated Unit Testing • What is unit testing? • Why unit test? • What do I test? • When do I test? • How do I test it?

10. What Should I Test? • Options • Test everything • Test critical code • Test frequently changing code • Test buggy code • Coverage has to be balanced with project timelines • Unit testing can save time on debugging • If you can't test everything make sure your testing time is well spent

11. Automated Unit Testing • What is unit testing? • Why unit test? • What do I test? • When do I test? • How do I test it?

12. When do I Test? • Option 1: Test Driven Development • Tests before code • Tests are written to fail, code is written to fix the tests • Works for some people, but just like cake, some people like it...some people prefer pie. • Option 2: Test Last • Tests are written on completed code. • When projects are rushed this tends to be the testing method • Tests risk being biased • Option 3: Test During • Test problem areas after they are identified

13. When do I Test? • Test as soon as you can in your process • Always create regression tests for new bugs • Don't become a victim of testing exhaustion • start small, get comfortable

14. Automated Unit Testing • What is unit testing? • Why unit test? • What do I test? • When do I test? • How do I test it?

15. How do I Test • Tests should be independant of the code being tested. • Tests should be easy to write • Tests should be easy to run • Tests should be easy to understand

16. PHPUnit • Created by Sebastian Bergmann • JUnit Port • Very Rich Feature Set • Mock Objects • Selenium Integration • Database Testing • Much Much More • Alternatives • SimpleTest • PHPT • test-more.php • lime

17. PHPUnit - Creating a Test Case <?phprequire_once('PHPUnit/Framework/TestCase.php');class SubStrTest extends PHPUnit_Framework_TestCase{    public function testSubstr()    {$this->assertEquals('o wo', substr('Hello world!', 4, 4));    }}?>

18. PHPUnit - Fixtures • Each test needs an established, predictable environment. • Tests should run completely independent of the results of previous tests. • setUp() - Common setup required for each test in the test case. • tearDown() - Hide your tracks!

19. PHPUnit - Call Order PHPUnit_Framework_TestCase::setUp();PHPUnit_Framework_TestCase::testMyCode1();PHPUnit_Framework_TestCase::tearDown();PHPUnit_Framework_TestCase::setUp();PHPUnit_Framework_TestCase::testMyCode2();PHPUnit_Framework_TestCase::tearDown();

20. PHPUnit - Fixtures <?phpclass BankAccountTest extends PHPUnit_Framework_TestCase{    protected $ba;    protected function setUp()    {$this->ba = new BankAccount;    }    public function testBalanceIsInitiallyZero()    {$this->assertEquals(0, $this->ba->getBalance());    }    protected function tearDown()    {$this->ba = NULL;    }}?>

21. PHPUnit - Testing Code • The basis of unit testing is making assertions regarding the expected results of a function. • assertEquals(), assertSame(), assertLessThan(), assertContains(), the list goes on and on and on and on ...

22. PHPUnit - Testing Code <?php$this->assertEquals('o wo', substr('hello world', 4, 4));$this->assertGreaterThan(3, 4);$this->assertTrue(TRUE);$this->assertNULL(NULL);$this->assertContains('value1', array('value1', 'value2', 'value3'));$this->assertFileExists('/tmp/testfile');$this->assertType('int', 20);$this->assertRegExp('/\d{3}-\d{2}-\d{4}/', '123-45-6789');?>

23. PHPUnit - Testing Code • The basis of unit testing is making assertions regarding the expected results of a function. • assertEquals(), assertSame(), assertLessThan(), assertContains(), the list goes on and on and on and on ... • A significantly fuller list: http://www.phpunit.de/pocket_guide/3.2/en/api.html#api.assert • Testing for Failure is also important: setExpectedException()

24. PHPUnit - Testing Code <?phprequire_once 'PHPUnit/Framework.php';class ExceptionTest extends PHPUnit_Framework_TestCase{    public function testException()    {$this->setExpectedException('Exception', 'exception message', 100);    }}?>

25. PHPUnit - Running a Test Usage: phpunit [switches] UnitTest [UnitTest.php]   --log-graphviz <file>  Log test execution in GraphViz markup.  --log-json <file>      Log test execution in JSON format.  --log-tap <file>       Log test execution in TAP format to file.  --log-xml <file>       Log test execution in XML format to file.  --coverage-html <dir>  Generate code coverage report in HTML format.  --coverage-xml <file>  Write code coverage information in XML format.  --test-db-dsn <dsn>    DSN for the test database.  --test-db-log-rev <r>  Revision information for database logging.  --test-db-prefix ...   Prefix that should be stripped from filenames.  --test-db-log-info ... Additional information for database logging.  --filter <pattern>     Filter which tests to run.  --group ...            Only runs tests from the specified group(s).  --exclude-group ...    Exclude tests from the specified group(s).  --loader <loader>      TestSuiteLoader implementation to use.  --repeat <times>       Runs the test(s) repeatedly.  --tap                  Report test execution progress in TAP format.  --testdox              Report test execution progress in TestDox format.

26. PHPUnit - Running a Test Usage: phpunit [switches] UnitTest [UnitTest.php]   --no-syntax-check      Disable syntax check of test source files.  --stop-on-failure      Stop execution upon first error or failure.  --verbose              Output more verbose information.  --wait                 Waits for a keystroke after each test.  --skeleton             Generate skeleton UnitTest class for Unit in Unit.php.  --help                 Prints this usage information.  --version              Prints the version and exits.  --configuration <file> Read configuration from XML file.  -d key[=value]         Sets a php.ini value.

27. PHPUnit - Test Suites • When you start amassing tests you need a way to organize them • Test Suites • Allow you to group test cases together so you can run one, all or a specific area's tests • Effective but somewhat cumbersome to set up

28. PHPUnit - Test Suites <?phpif (!defined('PHPUnit_MAIN_METHOD')) define('PHPUnit_MAIN_METHOD', 'AllTests::main');require_once 'PHPUnit/Framework.php';require_once 'PHPUnit/TextUI/TestRunner.php';require_once 'MyTest.php';require_once 'MyOtherTest.php';class AllTests{    public static function main()    {PHPUnit_TextUI_TestRunner::run(self::suite());    }    public static function suite()    {$suite = new PHPUnit_Framework_TestSuite('My Test Suite');$suite->addTestSuite('MyTest');$suite->addTestSuite('MyOtherTest');        return $suite;    }}if (PHPUnit_MAIN_METHOD == 'AllTests::main') AllTests::main();?>

29. PHPUnit - Test Suites • Using the xml config • You can specify a directory and a file mask • Test Case class names must follow specific format • Groups can still be set up (we'll talk about that soon) • Not as flexible as test suites • Incredibly simple to add new tests: Just drop the test in the appropriate directory

30. PHPUnit - XML Configuration Running a list of tests<phpunit>  <testsuite name="My Test Suite">    <directory suffix="Test.php">      mytests/directory    </directory>    <file>      mytests/dirtory/fileTest.php    </file>  </testsuite></phpunit>

31. PHPUnit - XML Configuration • So, how can I group tests at will when using XML to set up the test suite? • @group test annotation • --group, --exclude-group command line options

32. PHPUnit - XML Configuration <?phprequire_once('PHPUnit/Framework/TestCase.php');class SubStrTest extends PHPUnit_Framework_TestCase{/**     * @group builtins     */public function testSubstr()    {$this->assertEquals('o wo', substr('Hello world!', 4, 4));    }}?> >phpunit --configuration tests.xml --group builtins

33. PHPUnit - Annotations • Annotations can be an incredible time saver! • @test - Marks a particular method in a test case as a test • @dataProvider - Allow running the same tests using different data. • @expectedException - a convenient shortcut for setExpectedException • @assert - Helps in automatically generating tests

34. PHPUnit - Annotations • @test - Marks a particular method in a test case as a test • <?phprequire_once('PHPUnit/Framework/TestCase.php');class SubStrTestAnnote extends PHPUnit_Framework_TestCase{/**     * @test     */public function subStr1()    {$this->assertEquals('o wo', substr('Hello world!', 4, 4));    }}?>

35. PHPUnit - Annotations • @dataProvider - Allow running the same tests using different data. • <?phpclass DataTest extends PHPUnit_Framework_TestCase{    public static function provider()    {        return array(          array(0, 0, 0),          array(0, 1, 1),          array(1, 1, 3)        );    }/**     * @dataProvider provider     */public function testAdd($a, $b, $c)    {$this->assertEquals($c, $a + $b);    }}?>

36. PHPUnit - Annotations • @expectedException - a convenient shortcut for setExpectedException • <?phprequire_once 'PHPUnit/Framework.php';class ExceptionTest extends PHPUnit_Framework_TestCase{/**     * @expectedException Exception     */public function testException()    {    }}?>

37. PHPUnit - Skeleton Generator • When creating simple tests this can be incredibly useful. • <?phpclass Calculator{    public function add($a, $b)    {        return $a + $b;    }}?> • > phpunit --skeleton Calculator • PHPUnit 3.2.19 by Sebastian Bergmann.Wrote test class skeleton for "Calculator" to "CalculatorTest.php".

38. PHPUnit - Skeleton Generator • Result?/**     * @todo Implement testAdd().     */public function testAdd() {// Remove the following lines when you implement this test.$this->markTestIncomplete('This test has not been implemented yet.');    }

39. PHPUnit - Skeleton Generator > phpunit --verbose CalculatorTestPHPUnit 3.2.19 by Sebastian Bergmann.CalculatorTestITime: 0 secondsThere was 1 incomplete test:1) testAdd(CalculatorTest)This test has not been implemented yet./home/mike/phpunit/CalculatorTest.php:65OK, but incomplete or skipped tests!Tests: 1, Incomplete: 1.

40. PHPUnit - Skeleton Generator • Save some more time! • <?phpclass Calculator{/**     * @assert (0, 0) == 0     * @assert (1, 2) == 3     * @assert (2, 5) == 7     * @assert (3, 3) == 6     * @assert (4, 8) == 12     */public function add($a, $b)    {        return $a + $b;    }}?>

41. PHPUnit - Skeleton Generator • Result • /**     * Generated from @assert (0, 0) == 0.     */public function testAdd()    {$this->assertEquals(0,$this->object->add(0, 0)        );    }/**     * Generated from @assert (1, 2) == 3.     */public function testAdd2()    {$this->assertEquals(3,$this->object->add(1, 2)        );    }// ...

42. PHPUnit - Skeleton Generator > phpunit --verbose CalculatorTestPHPUnit 3.2.19 by Sebastian Bergmann.CalculatorTest.....Time: 0 secondsOK (5 tests)

43. PHPUnit - Skeleton Generator @assert (...) == X     > assertEquals(X, method(...))@assert (...) != X     > assertNotEquals(X, method(...))@assert (...) === X    > assertSame(X, method(...))@assert (...) !== X    > assertNotSame(X, method(...))@assert (...) > X      > assertGreaterThan(X, method(...))@assert (...) >= X     > assertGreaterThanOrEqual(X, method(...))@assert (...) < X      > assertLessThan(X, method(...))@assert (...) <= X     > assertLessThanOrEqual(X, method(...))@assert (...) throws X > @expectedException X

44. PHPUnit - Incomplete Tests TODO: finish slide

45. PHPUnit - Incomplete Tests Slide TODO: finish slidedon't worry...just a little irony

46. PHPUnit - Incomplete Tests <?phprequire_once 'PHPUnit/Framework.php';class SampleTest extends PHPUnit_Framework_TestCase{    public function testSomething()    {// Optional: Test anything here, if you want.$this->assertTrue(TRUE, 'This should already work.');// Stop here and mark this test as incomplete.$this->markTestIncomplete('This test has not been implemented yet.');    }}?>

47. PHPUnit - SkippedTests <?phprequire_once 'PHPUnit/Framework.php';class DatabaseTest extends PHPUnit_Framework_TestCase{    protected function setUp()    {        if (!extension_loaded('mysqli')) {$this->markTestSkipped('The MySQLi extension is not available.');        }    }}?>

48. Advanced Features Mock ObjectsDatabase TestingSelenium RCPHPUnderControlCome see me again! Same room, same day, 3:30 PM

49. Thanks http://phpun.ithttp://planet.phpunit.dehttp://digitalsandwich.comhttp://dev.sellingsource.comQuestions???