1 / 35

Story-Based Test Automation Using Free Tools

Story-Based Test Automation Using Free Tools. @ paul_gerrard. Paul Gerrard paul@gerrardconsulting.com. gerrardconsulting.com. Overview. Behaviour-Driven Development and similar approaches are still developer-focused Don’t engage testers as they should

natan
Download Presentation

Story-Based Test Automation Using Free Tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Story-Based Test Automation Using Free Tools @paul_gerrard Paul Gerrard paul@gerrardconsulting.com gerrardconsulting.com

  2. Overview • Behaviour-Driven Development and similar approaches are still developer-focused • Don’t engage testers as they should • Don’t support higher complexity requirements • Don’t scale to a larger requirements bases • In this session, we’ll cover: • Acceptance-driven development with free tools • How Story Platform for QA (a sub-set of Business Story Manager) and Robot Framework can be used to implement this approach. Intelligent Definition and Assurance

  3. Background • We won’t go into much detail of Behaviour-Driven Development • http://en.wikipedia.org/wiki/Behavior-driven_development for an intro • Requirements-test hierarchies: • Go to http://businessstorymanager.com • and watch the videos. Intelligent Definition and Assurance

  4. Behaviour-Driven Development using current tools Test Fixtures/ Test Step Defs. + Generated Test Code Story (structured) BDD Tool Test Outcomes System Under Test Test Tool executes tests Write the Code + Refactor the Code Intelligent Definition and Assurance

  5. BDD and tool support • A big benefit to developers is that BDD makes the Test-Driven Development approach easier • Behaviour-Driven Development: • A collaborative approach to specification • Uses Domain–Specific Language in specifications • Implements Ubiquitous Language • Currently, tools only support the automated testing part of the BDD regime. Intelligent Definition and Assurance

  6. Story Platform QA (BSM subset) Robot Framework Requirements Dictionary Keyword Definitions + Test Cases Story (structured) Test Export Test Outcomes System Under Test Test Library e.g. Webdriver executes tests • Test Status • Red • Amber • Green Write the Code + Refactor the Code Intelligent Definition and Assurance

  7. Business Story Manager and Test Frameworks Business Story Manager STORY: Feature Header Requirement Business Scenario Given“Search page” When “I find a book” And “I add to basket” Then “basket has my book” Just enough to clarify understanding Test Scenario Pre-conditions “Search page” Steps “enter search text” “click search button” “scroll down to find book” “select book” “view more information to verify” “select add to basket” Post Conditions/outcomes “Basket has selected book” Manually maintained Generated Technical Code Per Environment Keyword Definitions 1 per application Keyword Definitions 1 per Feature Story Based Test Cases Test Config. Test Log Example: Robot Framework Drivers System(s) Under Test

  8. How Does it Work? A walkthrough of the process using http://sp.qa

  9. The detail for the requirements selected on the left is shown here.

  10. The feature identifies a capability of the system described in the requirement Intelligent Definition and Assurance

  11. The scenario provides an illustration of a feature in use. Examples provide broader coverage of the feature. Intelligent Definition and Assurance

  12. NB: The colour coding highlights terms defined in the dictionary, and data items in scenarios. (The dictionary is available in the full product). This test scenario has been edited to provide more detail. These are the automated test keywords.

  13. The Lucky Numbers Application

  14. Lucky numbers features: • Registration • Submit ticket • View my Game Plays • System Status • Change my profile • Reset Password • etc. The numbers are submitted and validated and then the user is asked to confirm the ‘play’. Intelligent Definition and Assurance

  15. Intelligent Definition and Assurance

  16. Exporting Test Scenarios

  17. We’ve shown just one test scenario here for illustration. Normally, all test scenarios are exported.

  18. Dropbox asks for confirmation that you want to post the test case files to your account. Intelligent Definition and Assurance

  19. BSM lists all the files that have been posted to your Dropbox account. Intelligent Definition and Assurance

  20. Generated files and resource files The files generated by BSM have been synchronised with your Dropbox account These resource files correspond to your generated test case files. You maintain your keyword definitions here. Intelligent Definition and Assurance

  21. Robot script and resource file The Robot Script files with test cases are generated by Business Story Manager The tester maintains the resource file with the keyword definitions. Intelligent Definition and Assurance

  22. Tests running on a Linux VM that shares Dropbox Terminal window driving the tests System Under Test

  23. …and the same test on Windows 7 Intelligent Definition and Assurance

  24. Intelligent Definition and Assurance

  25. Results upload service Use a batch file or shell script to run the tests, then upload the results to BSM/SP.QA loadRobotOutput.py Python uploads the output file Intelligent Definition and Assurance

  26. Intelligent Definition and Assurance

  27. Business Story Manager Robot Framework Requirements Dictionary Keyword Definitions + Test Cases Story (structured) Test Export Test Outcomes System Under Test Test Library e.g. Webdriver executes tests • Test Status • Red • Amber • Green Write the Code + Refactor the Code Intelligent Definition and Assurance

  28. Behaviour-Driven Tooling • The essential architecture of BDD tools is that • A tool parses stories and generates test code • The test code can be used for TDD • The developer (almost always) writes the step definitions/fixture code and runs the tests • The stories, test code and steps are all source-controlled (in the developers’ library) • What we have done with SP.QA and Robot is allow non-techies to own this process and align it with stakeholder needs properly. Intelligent Definition and Assurance

  29. Summary • SPQA and Robot Framework combined: • Support requirements of any complexity • Implement keyword-driven test automation • Manage requirements, stories and tests to any scale • Acceptance-driven approach: • Traceable from requirements to tests and back • Automated story acceptance and regression testing • Build automated end-to-endacceptance tests with reusable keywords. Intelligent Definition and Assurance

  30. http://sp.qaStory Platform for QA SP.QA provides a subset of Business Story Manager functionality and is FREE-TO-USE.

  31. Visit SP.QA to access the free Story Platform for QA web service.

  32. Go to the Image Gallery to see the requirements-test process in pictures

  33. SP.QA Scope • What you get • Requirements module • Business Stories and Scenarios • Test Scenarios • Robot Framework Interface • FREE Service Limits • Limited to 100 business stories • 5 users, 1 application, 1 project

  34. BSM Scope • What you get • Stakeholder and Risk Management modules • Lean Task Management module • Dictionary and Library modules • Role-Based Access Control • All SP.QA Scope • Limitations • None • Commercials • £30 per user per month Intelligent Definition and Assurance

  35. Story-Based Test Automation Using Free Tools @paul_gerrard Paul Gerrard paul@gerrardconsulting.com gerrardconsulting.com

More Related