Succeeding with Behavior Driven Development (BDD) Testing and Automation

1. Succeeding with Behavior Driven Development (BDD) Testing and Automation Seattle Area Software Quality Assurance Group Oct 18, 2012 Alan Myrvold Google

2. Google Test Engineer, Ads Microsoft SDET in Office Security + Outlook Entrust Test manager, development manager, security assurance manager Cognos Tester, developer, test manager, development manager About me Feb 2011 - now 2005 - 2011 1998 - 2005 1988 - 1998

3. More about me

4. Buzzword bingo ATDD BDD Cucumber Cucumber - JVM Gherkin Jasmine SpecFlow

5. Buzzword bingo ATDD- acceptance test driven development BDD- behavior driven development Cucumber- a Ruby tool that supports BDD Cucumber - JVM - a Java tool that supports BDD Gherkin - the language used by Cucumber Jasmine - a javascript tool for BDD SpecFlow - a .NET tool for BDD

6. Cucumber Example Feature:Addition In order to avoid silly mistakes As amath idiot I wantto be told the sum of two numbers Scenario: Add two numbers Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen

7. Given / When / Then Given - precondition When - action Then - assertion on expected result And - same action as before

8. Cucumber step implementation @When("I have entered (.*) into the calculator") public void enterNumber(int number) { ... }

9. Scenario Outline Example Scenario Outline: Add two numbers Given I have entered <x>into the calculator AndI have entered <y>into the calculator When I press add Thenthe result should be <z>on the screenExamples:| x | y | z || 2 | 3 | 5 || 0 | -1 | -1 |

10. Why BDD? Clarifying requirements by example Demystifying automated tests by using English Demystifying repeated manual tests by emphasizing why and what to verify.

11. Some test types

12. How I used BDD at Microsoft Clarifying requirements in my test plan.

13. How we are using BDD at Google My group, DoubleClick Bid Manager, uses BDD for Java API-level system tests and repeated manual tests. Other groups use BDD tests for Java WebDriver tests. We share the same framework, developed internally.

14. Tools support + Books Tools: • Cucumber - Ruby http://cukes.info • Cucumber JVM https://github.com/cucumber/cucumber-jvm • SpecFlow - Binding business requirements to .NET codehttp://specflow.org Books • The Cucumber Book • The RSpec Book • Cucumber Recipies (beta, scheduled 3/7/2013) All books from Pragmatic Programmers, http://pragprog.com

15. When / Then elsewhere Mockito … a Java unit testing framework @Mock CalculationEngine engine; Calculator calculator = new Calculator(engine); when(engine.add(2, 2)).thenReturn(4); calculator.parse("2 + 2 ="); assertEquals("4", caclulator.getResult()); Compare the syntax to EasyMock: expect(engine.add(2, 2)).andReturn(4);

16. describe / it Rspec … a Ruby unit testing framework describe Calculator, "#basics"do it "return 4 for 2+2"do calc = Caculator.new calc.add(2, 2) calc.result.shouldeq(4) end end Jasmine … a Javascript unit testing framework describe("calc", function() { it("2+2 is 4", function() { expect(calc(2, 2).toEqual(4); }); });

17. BETTER When I add 50 and 70 Then the result is 120 Failure mode #1 - Too implementation dependent BAD Given I have entered 50 into the calculator And I have entered 70 into the calculator When I press add Then the result should be 120 on the screen

18. BETTER Then set the first 50 rows to "empty" Failure mode #2 - Programmatic Scripts BAD When I set x to 1 And while x < 50 And set row x to "empty"

19. BETTER When I log in as "bob" Failure mode #3 - Too low level BAD When I go to the login page And enter "bob" into the username field And enter "pass123" into the password field And click login Then I am logged in as "bob"

20. BETTER When I add 50 and 70 Then the result is 120 When I add 1e90, 0.1, and -1e90 Then the result is 0.1 Failure mode #4 - Not exploring interesting cases BAD When I add 50 and 70 Then the result is 120

21. BETTER @When("Set the budget to (.*)") public void setBudget(int amount) { ... } Failure mode #5 - Not using regex BAD @When("Set the budget to 100") public void setBudget() { ... }

22. BETTER @When("Set the budget to (.*)") public void setBudget(int amount) { ... } Failure mode #6 - Complex parsing logic BAD @When("(.*) the (.*) to (.*)") public void doAction(String action, String name, String value) { if (action.equals("set") && name.equals("budget") }

23. BETTER When I refresh all Failure mode #7 - Leaking code details BAD When I click the BTN-REFRESH-ALL button

24. BETTER Using supported public or test API's Failure mode #8 - Bad test architecture BAD Calling entry points in code that are fragile, or disappear

25. Success tips from me Use BDD for a small set of tests Focus on human readability, as a domain expert using the feature would describe a test

26. Success tips from The Cucumber Book, by Matt Wayne and Aslak Hellesoy DAMP beats DRY DAMP: Descriptive and meaningful phrases DRY: Don’t repeat yourself Declarative better than imperative Declarative: Given I am logged in Imperative: Log in as user “Bob”

27. Questions? http://testapprentice.com amyrvold@google.com