Michael Burnside Twitter: @ SQAEvangelist Blog: sqaevangelist.blogspot /

1. Effective Test Automation DEBT strategy and page/screen design pattern using data-driven testing paradigm for automated web/mobile app testing Michael Burnside Twitter: @SQAEvangelist Blog: http://sqaevangelist.blogspot.com/ Website: http://sqaevangelist.com/ Software Quality Assurance, Quality Engineering, and Web and Mobile Test Automation Expert 02 May 2014

2. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Table of Contents: • Mission Statement • Automated testing innovations and pros/cons • Decisions, Decisions….Which Technology to choose for automated testing investment? • Example of simple mobile app testing for iOS/Android apps. • Architecture • Test Technology Stack • Class diagrams • Run the tests demo • Q & A, Feedback

3. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Mission Statement The purpose of this presentation to identify test automation concepts, technology and design options while offering a learning example using a mobile application.

4. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Automated Testing Innovations: Pros/Cons (mobile) • Native language: ie, using the platform’s recommended programming language and test harness. • Pros: less dependency on third party libraries and support for each API release immediately. Minimal refactoring. • Cons: less flexibility and options possibly for test result reporting, trend analysis, and usually needs more “testability” from the actual app to be interacted with from the test app., requires test automation developers to proficient with code used (how hard would it be to find a QA person with Objective-C skills for iOS apps testing?)

5. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Automated Testing Innovations: Pros/Cons (mobile) – Part 2 2. Expressive language/keyword-driven: ie, using a proprietary expressive language to send directives to a mechanism that can interact with cross-platform UI’s and simulate human interaction use cases and use assertions. • Pros: Usually a small amount of training to learn and use language for testing. Possible contribution of non-technical resources to add tests. Some ability to test cross-platform. No testability changes on the app needed. • Cons: dependency of technology on software development API’s and OS’s which change rapidly and can break existing test automation. Example: Calabash

6. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Automated Testing Innovations: Pros/Cons (mobile) – Part 3 3. Data-Driven: ie, using many popular programming languages to interact with apps rendered on multiple platforms with lightweight software objects that interact with single mobile app screens and can test using simple workflows from files like Excel, java properties (.ini files), CSV. • Pros: tests. Ability to test cross-platform using the same code with framework supporting simple abstractions. Minimal effort to test another app on a different platform. Non-technical resources can create their own test testes for their test environments with their own accounts. No testability changes needed. • Cons: dependency of technology on software development API’s and OS’s which change rapidly and can break existing test automation. Usually need at least one skilled programmer on the QA team to facilitate the testing and have some object-oriented design knowledge. • Example: Appium

7. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Decisions, Decisions….Which Technology to choose for automated testing investment? What are the best decisions for my company organization considering the following information: • Do I want to pay for a product that allows for test automation for my app? • What are the investment risks? • When is my timeframe for X amount of code coverage? • What is the acceptable time/money investment for maintenance? • Am I comfortable with the dependencies on OS/API/Test harness dependencies? • Do I have enough technically capable resources on staff/available to invest a strategy for long-term maintenance investment? There are no right or wrong answers. Make the decision based on what you want to accomplish for short/long terms goals, or both. Regardless of what you choose, there is ALWAYS time/money investment in test automation.

8. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Example of simple mobile app testing for iOS/Android apps Let’s investigate a simple login application that is identical for iOS and Android mobile devices using the screen object design pattern using an entirely data-driven set of tests (no hard-coded tests). Desired: • Design and implement a set of automated tests using simple object-oriented design patterns that are directed and validated for acceptance criteria using data-driven files. • Easy capability to extend capability of testing the same app on a different mobile device OS. • Easy maintainability for lowest future cost and time effort.

9. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Architecture: There are 4 basic components to the automated testing solution: • Screen interfaces and implementation classes – interaction with the actual mobile app happens here. 3rd party api calls happen here. • Workflow classes – navigate from point A to point B in the application, execute a consistent set of operations • Base Screen and Test classes – utility encapsulating many widely-used methods in the language of choice and 3rd party method simplification. • Base functional test and screen classes per feature. Allows for customization of testing using separate accounts/users/data environment setup. • Test implementation classes using data-driven framework to parse files in CSV, Pipe-delimited, CSV

10. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Test Automation Technology Stack: • Java • Junit • TestNG • Appium • Web/Mobile Test automation framework (can be downloaded from SQAEvangelist.com – open source) • Xcode (iOS) • Android Studio ADT

11. Effective Test Automation Debt Strategy and Page/Screen Design Pattern • Create a single interface class for each screen that can be used for both iOS app and Android applications. • Create separate implementation classes (one for ios and other for android) • Create a single workflow that takes a parameter that indicates which screen implementation class to use. Both testing classes use the same workflow. • Data files drive the tests and can test positive and negative test cases.

12. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Example/Demonstration

13. Effective Test Automation Debt Strategy and Page/Screen Design Pattern Thanks! I sincerely hope this presentation was useful to you and that your company or organization implement the information and processes in this presentation to the fullest extent. If you have any feedback, please feel free to get in touch with me (via Twitter) at @SQAEvangelist. I would love to hear very constructive comments on how to improve this presentation and make it more effective.