NoVaTAIG April 7, 2010

1. NoVaTAIG April 7, 2010 Automated In-Browser Testing with Cucumber and Watir Rudy Regner Mike Perz Erik Scheirer Rasik Pandey

2. Rudy Regner • Introductory Notes • We’re all on linkedin.com. • Send us questions regarding any details presented tonight • Presentation Goals and Objectives • Mike Perz: Test Automation Framework • Erik Scheirer: End-to-End SDLC Architect • Rus Pandey: BDD Tools Expert

3. Our Situation • Writing regression tests for an app in production • Only required to support IE • Rails v1.2.3 • Precludes using Webrat latest version • ?? Could possibly have used earlier version ?? • why regression tests for earmarks? • Bug made it through manual testing • Encapsulate critical functionality in automated regression test scripts to decrease likelihood of this failure occurring again.

4. Mike Perz • How accomplished the automation framework • Challenges encountered

5. Our Learning Curve • Started with Watir • Easy to get up and running • Added Cucumber (RSpec) • Replaced Watir with Webrat and Selenium • Sought flexibility gain through re-use of Webrat to drive Selenium, Mechanize or straight Rails • Productivity dropped with Selenium • Watir can manipulate the DOM as objects. (you can iterate through the cells of an html table) • With Selenium you would have to find each cell individually. • Script debugging difficult without irb • You can attach to a browser window* (with Watir) • Converted existing Selenium code to Watir

8. Cucumber Feature snippet associated Step Definition

9. Scenario Outline example (similar to fitnesse tables)

10. Cucumber Test Summary Output (sample)

11. Once we switched back to Watir • At first our tests were dependent on each other • They needed to be run in a particular order. • We used Rake to accomplish this. • Open 4 browsers with 4 different users (Watir::IE.new_process vs. Watir::IE.new) • Now we can attach Watir to whichever browser session we need • Watir::IE.attach • We use SQLite to keep track of which user is associated with which browser session

12. Interesting side benefits to writing automated tests • Writing automated tests helped/made us understand the app better. • Found application bugs that were missed earlier

13. Other useful tools • Watircuke • IE Developer Toolbar • Firebug • FireWatir recorder:  www.itest2.com/downloads

14. Erik Scheirer • Spreading the Developer Joy • Benefits of a single executable Cucumber artifact end-to-end

15. Status - Your Organization • What Rules? Confusion? Sanity? • Even if its sane, it probably can be more so. • Some groups have managed a high degree of sanity. • But what most organizations are experiencing is ...

16. Chaos • little Trust between groups + infighting • bad communication • no ‘Process’ (or its limping, bleeding, out the door) • no time to implement anything new, constantly fighting fires instead

17. This approach • provides an end-to-end process via architecture, convention, and tools • is easy to implement • is easy to customize to exacting technical needs and group culture

18. This approach • builds better quality • builds more trust between technical and non-technical stakeholders • utilizes resources more effectively

19. Ingredients • CTBE - catch that bug early! • Unit tests, feature and black-box testing, via one single artifact • in this case an Agile approach (scrum) • artifact re-use creates self-reinforcing system

20. One Artifact, multiple uses • developer can create cucumbers as part of T/BDD, and also to capture ‘drive-by requirements’ • QA can create ‘cukes’ for regression tests • Cukes can be fed from dev. to QA or vice versa • Cukes can be used by Continuous Integration, and in Production monitoring

21. Next, the Process from multiple perspectives:

22. testing & daily work Iterations The daily grind Building Trust - spreading the joy Time The overall lifecycle Stakeholder interaction, design and architecture, includes non-technical

23. Conclusion • There is a way to achieve more success, better utilization of existing resources, and get more sleep; using a simple, adaptable architecture coupled with process, convention, and tools. • Did we mention ‘metrics’ and how these can be used to CTBE in this manner, too? • Its also all open-source, so its customizable to a great degree. • Thank you for listening.

24. Rus Pandey • How BDD would work • Other possible tools

25. Ideal BDD Process: • 1. Describe behaviour in plain text • 2. Write a step definition in cucumber • 3. Run and watch it fail • 4. Write code to make the step pass • 5. Run again and see the step pass • 6. Repeat 2-5 for all your functionality

26. Performance: • Since it takes 3 hours to run a test suite we should investigate using spork and --drb and testjour to distribute the test load if possible to speed up test runs.

27. Test Web apps in any language: Drive a full or headless browser using one of these • *Webrat – Ruby acceptance testing for web applications • *Capybara – Acceptance testing framework with a webrat-like API and support for multiple backends, including RackTest, Selenium, Celerity and Culerity • *Steam – Drives a fast headless browser with Javascript support. Support for the normal webrat step definitions, see Setting up Steam • *Testing PHP app using Webrat • *WebDriver – Drives IE, Firefox, Chrome • *Watir – Drives IE (Windows only), see: Setting up FireWatir • *FireWatir – Drives Firefox on Windows, OSX and GNU/Linux • *SafariWatir – Drives Safari (OS X only) • *ChromeWatir – Drives Google Chrome • *Celerity – Drives a fast headless browser with Javascript support. Examples here • *Culerity – For when you can’t run your app under jRuby • *Selenium – Runs any browser (any OS), see: Setting up Selenium • *Mechanize – Runs a headless browser (any OS)

28. Examples of Cucumber being used in the real world across various languages/platforms to test all sorts of things. • http://wiki.github.com/aslakhellesoy/cucumber/tutorials-and-related-blog-posts