1 / 29

Planning Test Automation

Planning Test Automation. (Wisely). Introductions. Thomas Messerschmidt. Avner Uzan. Test Automation Analyst. Test Automation Specialist Now @ Paramount Pictures . Disclaimer.

jolie
Download Presentation

Planning Test Automation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Planning Test Automation (Wisely)

  2. Introductions Thomas Messerschmidt Avner Uzan Test Automation Analyst Test Automation Specialist Now @ Paramount Pictures Planning Test Automation

  3. Disclaimer The views expressed in this presentation are those of the presenters & do not necessarily reflect the views of their current or past employers; their subsidiaries or affiliated companies! Planning Test Automation

  4. Planning Automation - Wisely • Premise • Set the hook • Basic Rules • Methodology: • Keyword-Driven Spreadsheet-Test Cases • Framework Construction • Scope: • Smoke, Functional, System, Performance • Prioritizing Test Construction • Best Kept Secrets (Trackers & Analysis Masters) • Test Automation Best Practices (Does Test automation Find Bugs?) Planning Test Automation 4

  5. Premise: Create maintainable automated test suites • Create a testing framework that will use easy-to-read spreadsheets to enumerate the actual test cases • Create & maintain generic & application specific functions & subroutines to support the automation Planning Test Automation 5

  6. Does Test Automation Find Bugs?

  7. Basic Rules • Plan your automation process thoroughly • Establish & follow naming conventions • Establish & persist a directory structure * • Build automation for platform independence • Solicit development’s cooperation to develop for testability • Expand your knowledge with other SMEs • MEASURE OR DIE Planning Test Automation 7

  8. Methodology • Smoke Testing – Keyword-driven Spreadsheets • Functional Testing – Keyword-driven Spreadsheets • System Testing – Functional Decomposition • Performance Testing – Weighted Functionality Planning Test Automation 8

  9. Zambelich Method Planning Test Automation 9

  10. Spreadsheet-Test Cases Planning Test Automation 10

  11. Generic Functions Planning Test Automation 11

  12. App Specific Functions Planning Test Automation 12

  13. Sample Keyword Action Planning Test Automation 13

  14. Parameter & Data File Planning Test Automation 14

  15. Smoke Test Objectives • Primarily validate builds • Open and close as many GUIs as possible • Some windows are acct data dependant • Manually test GUIs that could not be automated Smoke Test Execution Metrics • Covers 115/120 L1 GUIs - 105/110 L2 GUIs • Execution time: 1:25 / L1, 1:10 L2 • This was run 200+ times on the WR version • This has runs 200+ times on the QTP version • Currently demonstrates 100% GUI Pass rate Planning Test Automation 15

  16. GUI Functional GUI FunctionalObjectives • Validates the functionality of Each GUI • Verify screen functionality as per spec, NOT business processes • Validates objects exist on UI per spec • Verifies object operations, calculations & navigations • Verifies data formats ($-Amts, SSN, Phone #, etc.) • Verifies field/object data comparisons across screens • Verifies all error processing (invalid values, pop-ups, etc.) GUI Functional Scope • Targeted two languages Planning Test Automation 16

  17. System Test Use Functional Decomposition System Test Objectives • Focus on core functionality – align w/ project team &/or customer • Regression test regular release & special projects • Future plans include converting our system test to BPT & auto. Scope Execution Metrics • 200 scripts of which about 175 are active - take about 4-5 days • 50 full suite runs, 10,000 tests run over 5 year period Planning Test Automation 17

  18. Performance Test Planning Test Automation

  19. Prioritize Test Construction • Target low hanging fruit to get early ROI • Prioritize &build common functions first • Create tests for most frequented GUIs Planning Test Automation 19

  20. Optimizing Automation • Optimize automation for maximum coverage • Increase number of executing machines • The downside • Brake automation into “byte-size” pieces • If one function can do the work of two Planning Test Automation 20

  21. Other Considerations • Number of test platform / environments • Availability of test platform (hours/day) • Comparability of test environment to prod. • Environment refresh • Test data availability / harvesting (method of) • Test data accuracy • Product or application stability Planning Test Automation 21

  22. Trackers (Measure or Die) Planning Test Automation 22

  23. Analysis Masters • Demo Analysis Masters Planning Test Automation 23

  24. Performance Analysis Master Planning Test Automation 24

  25. Automation Best Practices • Setup directory structure & use it consistently • Define & establish filing conventions • Publish changes to subroutines & functions. • Keep test assets under version control • Choose automation tools carefully based on compatibility to your development platform, resource knowledge, budget. • Include resource, time & process allowances for test asset maintenance. • Keep an inventory list of functions & routines easily accessible. • If possible cross-reference use of functions & subroutines in scripts to more predicatively understand the impact changes to these assets might have. • Prioritize tests to be automated carefully to avoid marginalizing ROI Planning Test Automation

  26. Automation Best Practices2 • Get management buy-in & commitment early • Set realistic expectations – automation is not just the push of a button • Identify a set of relevant metrics & measure regularly & consistently • Include regular progress reviews to ensure that the automation team is in fact focused on the business or customer needs • Share the wealth – the more people use it the richer you’ll be! Additional References: TOTALLY DATA-DRIVEN AUTOMATED TESTING – Keith Zambelich http://automation.org.uk/downloads/documentation/white_papers-totally_data_driven_automated_testing.doc Keys to Functional Test Automation Success, by Thomas Murphy / The Gartner Group - Thanks to Microfocus an HP Partner for the link http://www.gartner.com/technology/media-products/reprints/microfocus/vol4/article4/article4.html LoadRunner Test Analysis Master – A. Uzan http://www.youtube.com/watch?v=m860IurgHM8 Planning Test Automation

  27. Automation Demo • Demo Automation (time permitting) Planning Test Automation 27

  28. Avner Uzan: qaanalysis@ca.rr.com Thomas Messerschmidt: qa4quality@gmail.com Contact us: Planning Test Automation

  29. Parting Thought On Automation Why Automate?Automation, fundamentally, is the mechanization of a manual task. Love, education & human interaction should not be automated. However, there is no evil in taking a manual process & allowing a machine to do that which would take a person longer, work a person harder or take the pleasure out of doing a task over & over. However, if by automating something, the existence of life, the values of morality, the sensibilities of one’s existence or the livelihood of those who would otherwise perform the task manually is usurped, than automation has defeated it’s very purpose. Automation should be carefully conceived so that future generations are not burdened with undoing the damage of unwitting or well intended but misguided endeavors. Planning Test Automation

More Related