O utline

1. 1

2. Outline • Introducing BDD • BDD principles • BDD tools • What is Concordion? • Why Concordion? • Concordion in use • Getting along with Maven, Ant and Eclipse • BDD with Concordion • Technique • Common Smells • Resources 2

3. Introducing BDD 3

4. What is BDD? BDD: • Stands for behavior-driven development. • Was originally conceived in 2003 by Dan North as a response to TDD. • Is an evolution in the thinking behind Test-Driven Development and Acceptance Test-Driven Planning. • Aims to help focus development on the delivery of prioritized, verifiable business value by providing a common vocabulary (also referred to as a Ubiquitous Language) that spans the divide between Business and Technology. • Relies on the use of a very specific (and small) vocabulary to minimize miscommunication and to ensure that everyone – the business, developers, testers, analysts and managers – are not only on the same page but using the same words. 4

5. BDD principles • Test method names should be sentences It can do at least some of your documentation for you, so start to write test methods that are real sentences. What’s more, when you write the method name in the language of the business domain, the generated documents make sense to business users, analysts, and testers. • A simple sentence template keeps test methods focused Start test method names with the word “should.” This sentence template – The class should do something – means you can only define a test for the current class. This keeps you focused. If you find yourself writing a test whose name doesn’t fit this template, it suggests the behavior may belong elsewhere. If a class is doing more than one thing, take it as an indication that you should introduce other classes to do some of the work.

6. BDD principles (continued) • An expressive test name is helpful when a test fails If you’re changing code and cause a test to fail, you can look at the test method name and identify the intended behavior of the code. Typically one of three things happen: • You had introduced a bug. Solution: Fix the bug. • The intended behavior is still relevant but have moved elsewhere. Solution: Move the test and maybe change it. • The behavior is no longer correct – the premise of the system have changed. Solution: Delete the test. The latter is likely to happen on agile projects as your understanding evolves. Unfortunately, novice TDDers have an innate fear of deleting tests, as though it somehow reduces the quality of their code. A more subtle aspect of the word should becomes apparent when compared with the more formal alternatives of will or shall. Should implicitly allows you to challenge the premise of the test: “Should it? Really?” This makes it easier to decide whether a test is failing due to a bug you have introduced or simply because your previous assumptions about the system’s behavior are now incorrect.

7. BDD principles (continued) • “Behavior” is a more useful word than “test” BDD gives answers to some of the most vague TDD questions: • What to call your test is easy – it’s a sentence describing the next behavior in which you are interested. • How much to test becomes moot – you can only describe so much behavior in a single sentence. • When a test fails, simply work through the process described above – either you introduced a bug, the behavior moved, or the test is no longer relevant.

8. BDD principles (continued) • Determine the next most important behavior A really useful way to stay focused is to ask: What’s the next most important thing the system doesn’t do? This question requires you to identify the value of the features you haven’t yet implemented and to prioritize them. It also helps you formulate the behavior method name: The system doesn’t do X (where X is some meaningful behavior), and X is important, which means it should do X; so your next behavior method is simply: public void shouldDoX() { // ... } Now we have an answer to another TDD question, namely where to start.

9. BDD principles (continued) • Requirements are behavior, too • BDD provides a “ubiquitous language” for analysis

10. BDD Tools 10

11. ASSpec - ActionScript 3 BDoc - Extracting documentation from unit tests, supporting behaviour driven development BDD in Python is core module doctest Bumblebee - Extract documentation from JUnit tests with support for adding text, code-snippets, screenshots and more. Puts focus on the end-user. beanSpec - Java cfSpec - ColdFusion CSpec - C dSpec - Delphi Concordion - a Java automated testing tool for BDD that uses plain English to describe behaviors. Cucumber - Plain text + Ruby. Works against Java, .NET, Ruby, Flex or any web application via Watir or Selenium. easyb - Groovy/Java EasySpec - Groovy, usable in Java. Developer also working on Perception a tool for doing Context/Specification reporting for many different tools. GSpec - Groovy Instinct - Java JavaStubs - Java - BDD framework supporting partial-mocking/method stubbing JBehave - Java JDave - Java JFXtras Test - JavaFX JSpec - JavaScript JSSpec - JavaScript NBehave - .Net NSpec - .Net NSpecify - .Net NUnit - Another implementation of BDD framework in .Net with focus on specification testing PHPSpec - PHP Pyccuracy - Behavior-driven framework in Python. Pyhistorian - General purpose BDD Story Runner in Python (internal DSL, not plain-text) RSpec - Ruby ScalaTest - Scala specs - Scala spec-cpp - C++ Specter - Another implementation of BDD framework in .Net with focus on specification readability StoryQ - .Net 3.5, can be integrated with NUnit to provide both specification readability and testing tspec - Groovy (Thai syntax) BDD Tools (continued)

12. What is Concordion? 12

13. Key Features Concordion is an open source tool for writing automated acceptance tests in Java • Powerful, yet simple to use • Concordion integrates directly with JUnit. • Highly readable tests • Concordion acceptance tests are so readable they can double up as system documentation. And, since the tests are linked to the system, you know the documentation is always up-to-date. • Separates tests from implementation • Tests that include a lot of implementation detail lock you into that implementation. Concordion helps you to document the logic and behavior of your system in a way that does not lock you in.

14. What's special about it? • Plain English specifications • Rather than forcing product owners to specify requirements in a specially structured language, Concordion lets you write them in plain English using paragraphs, tables and proper punctuation. This makes the specifications much more natural to read and write, and helps everyone to understand and agree about what a feature is supposed to do. • Always bang up-to-date • Concordion specifications are active. Behind the scenes, they are linked to the system under test and therefore do not go out-of-date. If a change is made to the system's behavior then the tests associated with the relevant specification will fail and let you know. • Complex behaviors can be decomposed • Each question at the bottom of the specification links to another active specification and so on until you have either run out of questions or the answer is "Out of Scope". In this way, a complex behavior can be broken down into small, focused and easily understood requirements.

15. Concordion in use • Specifications are written in simple HTML. Developers instrument the concrete examples in each specification with commands (e.g. "set", "execute", "assertEquals") that allow the examples to be checked against a real-life system. • The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification and acts as a buffer between the specification and the system under test. The fixture is also a JUnit test case, so it's easy to run and integrate into an automated build. The results of running the examples are exported with green and red indicating successes and failures. • Some example instrumentation: <p> When <span concordion:set="#firstName">Bob</span> logs in a greeting <span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span> should be displayed. </p>

16. Concordion commands • A Concordion active specification consists of two parts: • a well-formed XHTML document describing the functionality • fixture code written in Java that finds concrete examples in the document and uses them to verify the system under test. Both files must be in the same package. • In order for the magic to happen, the document must first be instrumented with commands. • Concordion commands are specified as attributes on elements in the XHTML document. Web browsers ignore attributes that they don't understand, so these commands are effectively invisible. • The commands use a "concordion" namespace defined at the top of each document as follows: <html xmlns:concordion="http://www.concordion.org/2007/concordion">

17. concordion:assertEquals <html xmlns:concordion="http://www.concordion.org/2007/concordion"> <body> <p concordion:assertEquals="getGreeting()">Hello World!</p> </body> </html> packageexample; importorg.concordion.integration.junit4.ConcordionRunner; importorg.junit.runner.RunWith; @RunWith(ConcordionRunner.class) public classHelloWorldTest{ publicString getGreeting() { return "Hello World!"; } } By default, Concordion outputs to the directory specified by the system property java.io.tmpdir.

18. concordion:set <html xmlns:concordion="http://www.concordion.org/2007/concordion"> <body> <p> The greeting for user <span concordion:set="#firstName">Bob</span> will be: <span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span> </p> </body> </html> packageexample; importorg.concordion.integration.junit4.ConcordionRunner; importorg.junit.runner.RunWith; @RunWith(ConcordionRunner.class) public classHelloWorldTest { publicString greetingFor(String firstName) { return "Hello " +firstName+ "!"; } }

19. concordion:execute The execute command has three main uses: • Executing an instruction with a "void" result. • Executing an instruction with an object result (to allow multiple properties of the object to be checked). • Handling unusual sentence structures.

20. concordion:execute. Part 1 Executing an instruction with a void result • It can occasionally be useful to execute an instruction that sets up some system state. Every time you do this, however, alarm bells should ring in your head and you should question yourself to make sure that you are not inadvertently writing a script instead of a specification. E.g. a call to clearDatabase() would be a blatant misuse. • As a rule of thumb, methods with a void result called from an execute should start with the word set or setUp. E.g. setUpUser(#username). <html xmlns:concordion="http://www.concordion.org/2007/concordion"> <body> <p> If the time is <span concordion:execute="setCurrentTime(#TEXT)">09:00AM</span> then the greeting will say: <span concordion:assertEquals="getGreeting()">Good Morning World!</span> </p> </body> </html> #TEXT special variable contains the text of the current element.

21. concordion:execute. Part 2 Executing an instruction with an object result Sometimes you need to check more than one result of a behavior. For example, here we want to check that both the first name and the last name are correctly extracted from the full name: <html xmlns:concordion="http://www.concordion.org/2007/concordion"> <body> <div class="example"> <p> The full name <span concordion:execute="#result = split(#TEXT)">John Smith</span> will be broken into first name <span concordion:assertEquals="#result.firstName">John</span> and last name <span concordion:assertEquals="#result.lastName">Smith</span>. </p> </div> </body> </html>

22. concordion:execute. Part 2(continued) package example; import org.concordion.integration.junit4.ConcordionRunner; import org.junit.runner.RunWith; @RunWith(ConcordionRunner.class) public class SplittingNamesTest { public Result split(String fullName) { Result result = new Result(); String[] words = fullName.split(" "); result.firstName = words[0]; result.lastName = words[1]; return result; } class Result { public String firstName; public String lastName; } }

23. concordion:execute. Part 3 Handling unusual sentence structures One of the great things about Concordion is that when you're writing the specifications you do not have to worry about how you're going to instrument it. You can just concentrate on making the document as readable as possible. <p> Upon login, the greeting for user <span concordion:set="#firstName">Bob</span> will be: <span concordion:assertEquals="greetingFor(#firstName)">Hello Bob!</span> </p> vs <p concordion:execute="#greeting = greetingFor(#firstName)"> The greeting "<span concordion:assertEquals="#greeting">Hello Bob!</span>" should be given to user <span concordion:set="#firstName">Bob</span> when he logs in. </p> The execute command is designed to process commands on its child elements in a special order. First of all it processes any child set commands then it runs its own command, then any child execute commands and finally any child assertEquals commands.

24. concordion:execute on a <table> When you want to show several examples of a behavior, repeating the same sentence structure over and over again probably isn't going to be very nice to read. It would be better to use a table. <table concordion:execute="#result = split(#fullName)"> <tr> <th concordion:set="#fullName">Full Name</th> <th concordion:assertEquals="#result.firstName">First Name</th> <th concordion:assertEquals="#result.lastName">Last Name</th> </tr> <tr> <td>John Smith</td> <td>John</td> <td>Smith</td> </tr> <tr> <td>David Peterson</td> <td>David</td> <td>Peterson</td> </tr> </table>

25. concordion:verifyRows Sometimes you want to check the contents of a collection of results returned from the system. In the Fit Framework you might use a RowFixture. In Concordion, you use the verifyRows command. <table concordion:execute="setUpUser(#username)"> <tr><th concordion:set="#username">Username</th></tr> <tr><td>john.lennon</td></tr> <tr><td>ringo.starr</td></tr> <tr><td>george.harrison</td></tr> <tr><td>paul.mccartney</td></tr> </table> <p>Searching for "<b concordion:set="#searchString">arr</b>" will return:</p> <table concordion:verifyRows="#username : getSearchResultsFor(#searchString)"> <tr><th concordion:assertEquals="#username">Matching Usernames</th></tr> <tr><td>george.harrison</td></tr> <tr><td>ringo.starr</td></tr> </table> The syntax for a verifyRows command is: #loopVar : expression Where expression returns an Iterable object with a predictable iteration order, (e.g. a List, LinkedHashSet or a TreeSet). And #loopVar provides access to the current object during iteration and allows the assertEquals method to check its value.

26. concordion:verifyRows(continued) The order of the items in the table being verified must match the iteration order of the items returned by the expression. You may need to sort the items to ensure they are in a known and consistent order. In our example, we are using alphabetical order ("george" before "ringo"). @RunWith(ConcordionRunner.class) public class PartialMatchesTest { private Set<String> usernamesInSystem = new HashSet<String>(); public void setUpUser(String username) { usernamesInSystem.add(username); } public Iterable<String> getSearchResultsFor(String searchString) { SortedSet<String> matches = new TreeSet<String>(); for (String username : usernamesInSystem) { if (username.contains(searchString)) { matches.add(username); } } return matches; } }

27. Getting along with Maven, Ant and Eclipse The simplest configuration to run Concordion tests with maven. <build> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> <configuration> <source>1.5</source> <target>1.5</target> </configuration> </plugin> <plugin> <artifactId>maven-surefire-plugin</artifactId> <configuration> <systemProperties> <property> <name>concordion.output.dir</name> <value>target/concordion</value> </property> </systemProperties> </configuration> </plugin> </plugins> </build>

28. Getting along with Maven, Ant and Eclipse (continued) The simplest configuration to run Concordion tests with Ant. Since Concordion is built on top of JUnit it takes no additional effort to run Concordion tests with JUnit. The same is true for the Eclipse IDE as well as any IDE that supports JUnit. <junit fork="yes" forkmode="once" printsummary="yes" haltonfailure="yes" showoutput="yes"> <jvmarg value="-Dconcordion.output.dir=build/concordion-output"/> <classpath> <path refid="compile.classpath"/> <pathelement location="build/classes"/> </classpath> <formatter type="plain" /> <batchtest todir="build/test-output"> <fileset dir="specs"> <include name="**/*Test.java"/> <exclude name="**/Abstract*"/> </fileset> </batchtest> </junit>

29. BDD with Concordion 29

30. Technique

31. Write specifications, not scripts • Scripts over-specify • Test scripts are a list of instructions to be followed. For example: • Clear database • Load database from "sample-data.sql" • Start webserver • Open URL: http://localhost:8080/myapp • Enter username: admin • Enter password: admin1 • Click the "Login" button • Click the "User Administration" link • Click the "Create User" button: • Enter name: John Smith • Enter username: john • Enter password: john99 • Click the "OK" button • Click the "Logout" link • Enter username: john • Enter password: john99 • Click the "Login" button • Check page contains text: Hello John!

32. Write specifications, not scripts(continued) • Concealed inside the script is a behavior that the test is trying to demonstrate. But because the requirement is not explicit, it's hard to know what it is (e.g. "Can we change this link to a button, or is the link part of the requirement?"). • Scripts also tend to suffer badly from duplication. If we introduce an extra step into the authentication process then all the scripts that mention logging-in will need modification. This is exactly the kind of duplication that programming languages are designed to address and why plain English is not a good language for scripting.

33. Write specifications, not scripts(continued) • Specifications give you freedom • Specifications tell you the requirements explicitly. They are written at a higher level of abstraction to test scripts. For example: When John logs in, a greeting "Hello John!" is displayed. • This can be implemented in multiple ways. All of the details about how to test the requirement are hidden inside the fixture code where they can be refactored as the system evolves.

34. Specifications should be stable • The specifications themselves should rarely change. Even on agile projects, though new behaviour is added frequently, existing behaviour is normally maintained from iteration to iteration.

35. Evolve a domain-specific language As you write the fixture code, and refactor it to remove duplication, you'll find you gradually build up a scripting API – a domain-specific language (DSL) – that lets you manipulate the system under test. Eventually, the fixture code will become very stable too.

36. Isolate behaviours • Each active specification should have a narrow focus and cover a single behavior as independently as possible from other behaviors. For example, one specification might describe how text searches are case-insensitive, another will describe how date searches work, and another will describe how search results are to be presented (what data is displayed). The idea is to keep each specification very simple and to avoid overlaps, so that we can change the specification about the way search results are presented without having to make changes to the other specifications. • If you want to test a combination of behaviors, write a separate specification of the behavior for the combination. But always write the specifications for the individual behaviors first. When you combine behaviors there is a penalty in terms of complexity – of both the specification and the fixture code to support it.

37. Think "Given-When-Then" • This is an excellent way of structuring the concrete examples in the specifications, and getting into the mindset of specifying instead of scripting. Given (some context) When (something happens) then (some behavior) Example: Given a user called John; When John logs in, then a greeting is displayed saying "Hello John!". Once you've written or thought about your behavior using the Given-When-Then template, you can reword the sentence to make it less clunky. For example, in this case we can deduce that John is a user from the phrase "John logs in", so we can drop the "Given" part and write something more readable like this: When John logs in, a greeting "Hello John!" is displayed.

38. Common Smells

39. Existing specifications are often changed • You're tied to an implementation • If you find you're having to change the contents of specifications/instrumentation, on a regular basis, this is a strong indicator that the specifications are too closely coupled to the implementation. • So, pretend there are multiple implementations • The solution is to describe the underlying behavior in a more abstract way. It might help to imagine that there are several different implementations - a web application, a Swing application, and a command-line executable, for example. Only describe behavior that is required so that you leave as many options open as possible for alternative implementations.

40. Lots of "execute" commands • You're writing a script • If you have lots of "execute" commands you are probably writing a script instead of a specification. • So, hide the scripting in the fixture • Avoid describing the steps of how to test the behavior and simply state the context (as a sentence), and the behavior you expect to see. Don't explain how to get into that context, don't explain how to perform the operation and don't explain how to extract the results. All of these things are implementation-specific and should be hidden in the fixture code.

41. Complicated instrumentation • You're testing too much in one go • As a rule of thumb, well-written fixtures should have no more than three public methods and no method should have more than one or two parameters. Complicated instrumentation is usually a sign of trying to test too many things at once. • So, decompose the behavior • Focus on one tiny behavior at a time. Break the behavior into smaller and smaller pieces until you cannot break it down any further. Then write separate specifications for each piece.

42. Complicated fixture code • Your fixture code is verbose and hard to follow • All the scripting should be done in the fixture code, but this can make the code seem complicated. • So, create classes to help with the scripting • Begin to create objects to help with the scripting activity - push the implementation details into them. Eventually you'll create a little domain specific language (DSL).

43. Examples all have the same structure • Your examples are too generic • If the examples in your specifications look very similar (i.e. they have the same kind of context set-up, the same kind of checks etc.) this is a strong sign that the examples are too generic. • So, focus the examples more carefully • The examples should demonstrate the particular behavior you are describing and should not include any irrelevant details. Push everything you can into the fixture code. The layout above is "one size fits all". It contains lots of context that is not relevant to the behavior we want to show. Focusing the examples improves their clarity an reduces duplication.

44. Resources

45. Useful links • Dan North's article introducing BDD: http://dannorth.net/introducing-bdd • Introduction to Behavior Driven Development:http://behavior-driven.org/ • In pursuit of code quality: Adventures in behavior-driven development by Andrew Glover:http://www.ibm.com/developerworks/java/library/j-cq09187/index.html • The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends: http://www.pragprog.com/titles/achbd/the-rspec-book • Concordion site:http://www.concordion.org/ • Concordion page at google code:http://code.google.com/p/concordion/w/list