There is an A for everything

1. There is an A for everything • Theoretical side of things • Definitions • Vocab • ISTQB • Good practices • Things to remember • Lessons to learn • Practical (realistic ) side of things • Activities and their benefits • Examples

2. Summary : • Define test automation framework • Types of automation frameworks • Create your own framework • Focus Points • Layer your framework Test Automation Frameworks Roxana Cracana December 4th, 2013

3. What is a Test Automation Framework?

4. Types of automation testing frameworks Automation Testing Frameworks Agile Frameworks

5. Record and Playback is not an framework What is… • Capture interaction with system and replay it • Popular approach among commercial automation tools Advantages • Very easy and fast to create initially • No programming skills needed • Easiest way to learn the features of the testing tool Disadvantages • Does not test anything unless checkpoints added • Very fragile – often single change in UI can break all tests • No modularity or reuse • Hard to maintain

6. Linear testing framework What is… • Non-structured scripts interact directly with the SUT • Also produced by capture and replay tools Advantages • Fast way to create script • Automation expertise not required, just tool knowledge Disadvantages • No modularity or reuse • Test data is hard coded into the script • Very fragile – one change in the system may break all scripts • Plenty of separates scripts • Never a good basis for large scale automation Test script 1 SUT Test script 2

7. Modular testing framework What is… • Interaction with the SUT done by functions in a test library Advantages • Higher level of code reuse • Easier script maintenance Disadvantages • Building test library requires initial effort: takes time and programming skills • Test data embedded into scripts Test script 1 Test library SUT Test method 1 Test method 2 Test script 2 Test method 3

8. Data-driven testing framework What is… • Test data taken out of test scripts – usually externalized in tabular format Advantages • Changes to the test scripts do not affect the test data • Test scripts can be executed with multiple sets of data Disadvantages • Creating scripts requires programming skills and time • Initial effort creating data parsers can be high • May be an overkill for simple needs Test Data 1 Test script 1 Test library SUT Test method 1 Parser Test method 2 Test script 2 Test Date 2 Test method 3

9. Keyword-driven testing framework What is… • Utilizes data tables and self explanatory keywords to indicate the actions performed on SUT Advantages • Provides high code re-usability • Test tool independent • Tests can be designed with or without SUT • Very good for large scale use Disadvantages • Initial effort can be really high Test script 1 Test library SUT Keyword 1 Keyword 2 Test script 2 Keyword 3

10. Hybrid testing framework What is… • Is the combination of modular, data-driven and keyword driven testing frameworks Advantages • Integrates the advantages of all the other frameworks. Disadvantages • Very complex

11. Which type of framework to choose? Create your own framework!

12. Focus point • Good testing frameworks have layers • Layer 1: Control level • Layer 2: Business level • Layer 3: Test level Build layers over existing automation tools!

13. Focus point • Test error and failure handling • Decide when to fail your test • Decide when to fail your test suite • Think about tests setup and tests recovery Soft assertion vs Hard assertion

14. Focus point • The key artefact of a testing framework is REPORTING • Summary report • Detailed report • Reports repository • Traceability • Metrics Integrate early logging and reporting into the framework!

15. Focus point • Object Repository • Also called Application map • Stores AUT objects • Multiple ways to represent it Identify the best way to represent OR!

16. Focus point • Continuous automation • Parallelize test execution • Distribute test execution • Schedule test execution • Automate test machine setup • Automate test precondition • Automate notifications Integrate test execution into a continuous integration tool!

17. Layer your framework – Control Level(1)

18. Layer your framework – Control Level(2)

19. Layer your framework – Control Level(3) • Dropdown actions • Edit actions • Select value • Select index • Verify selected value • Verify all values • Verify value exist • Verify value not exist • Verify number of values • Verify values are sorted • Type value • Clear value • Append value • Enter value • Verify value • General actions • Verify is displayed • Verify is not displayed • Verify is enabled • Verify is disabled • Verify background colour • Verify label • Verify tooltip

20. Layer your framework – Control Level(4) Edit control – Verify Value method Logging Error handling : Hard assertion

21. Layer your framework – Business Level(1)

22. Layer your framework – Business Level(2) Business Unit: HomePage Business Unit: Header Business Unit: Start Session Business Unit: Register

23. Layer your framework – Business Level(3) Application MAP for Start Session Edit_Email Edit_Password CheckBox_AutoLogin Link_ClickHere Text_Errors Button_StartSession Button_Register

24. Layer your framework – Business Level(4) Start Session Business Unit

25. Layer your framework – Test Level(1)

26. Layer your framework – Test Level(2) Test Case vs. Test Script Keyword driven test script

27. Layer your framework – Test Level(3) Data driven test script

28. Conclusion A hybrid automation testing framework