1 / 57

PREDICT Model for Test Automation

PREDICT Model for Test Automation. Does it sound familiar to you?. Organization has procured test automation tools Management expectations are high Multiple test automation project initiatives started Enthusiasm, commitment and motivation declined with time

lise
Download Presentation

PREDICT Model for Test Automation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PREDICT Model for Test Automation

  2. Does it sound familiar to you? Organization has procured test automation tools Management expectations are high Multiple test automation project initiatives started Enthusiasm, commitment and motivation declined with time Script maintenance had become a significant challenge Test automation tools are back on shelf

  3. Testing and test automation requires same skills Capture/Replay is a successful test automation approach Test automation is an integral part of the over all test process Test automation is effective in solving regression testing issues Every test and test type can be automated Test automation development is software development Test automation projects can be delivered by personnel with strong project management skills Spot Quiz

  4. Test automation solution approach Essential building blocks for successful test automation solutions

  5. PREDICT Model

  6. PREDICT Model Structured methodology Multi stage process consisting of seven components Plan Rationalize Estimate Design Implement Consolidate Transition Step-wise approach Easy to apply

  7. PREDICT Model - Plan Plan Rationalize Estimate Transition Design Consolidate Implement

  8. Need for test automation project What do you consider to start a project ? Test related checks Objective Scope Documentation Test ware Organization related checks Investments Expectations Resource availability Technology related checks Stability of SUT Test tool suitability

  9. Need for test automation project Are you getting bang for your buck ? Costs Licensing Framework Implementation Maintenance factor Benefits Saved manual effort per test run Number of test runs Re-usability factor Amortizing benefits Life cycle of the application/product Opportunities for test automation

  10. Plan of approach Introduction Objective Pre-requisites Set-up pilot project Activities Planning Deliverables Organization Resources

  11. PREDICT Model - Rationalize Plan Rationalize Estimate Transition Design Consolidate Implement

  12. How do we rationalize requirements ? 5 P factors Project Process Product People Price Test suite attributes Building trade-offs based on project context

  13. Project Objective and scope Clear objectives in terms of time, money and quality Strategy Test automation strategy fitment with manual testing strategy Build Build frequency Configuration management Bad experiences User interface Stability Cycle time Regression possible

  14. Process Quality documentation Manual test scripts, documentation Development approach Phases clearly defined, how early can test automation start Test practice New builds tested before delivering to QA Regression moments Naming conventions User Interface conventions

  15. Product Stability Base platform stability Testability Degree of interaction between the tool and product under test Test tool integration Integration between test management tool and configuration management Test oracles Interaction with non UI components,database Automation framework Availability of frame work to reuse

  16. People Organization readiness Management expectations Resource availability Machine, Licenses and people with right skills Non- programming resources Framework inclined towards non-programming resources QA population Ratio of development to QA Collaboration Collaboration between QA and Dev Technical skill set Test engineers with technical skills

  17. Price Return on Investment Opportunities of reusing test automation investment Reusability of Investment Are there any products which are tested in a similar way Business Plan Is there a business plan to realize benefits of Test Automation

  18. Test suite attributes Maintainability Reusability Reliability Usability Portability

  19. Which is best test automation regime ? IT DEPENDS ON THE PROJECT CONTEXT

  20. PREDICT Model - Estimate Plan Rationalize Estimate Transition Design Consolidate Implement

  21. Test automation estimation challenges Test organization maturity Scope of requirements Framework Skill level Test case complexity Domain complexity Test tool Testability

  22. Estimation techniques Delphi Gather estimates from each team member Ask high and low estimator to explain estimates Repeat twice, then use average Three point method Have team estimate best, worst case and expected case Use expected case Variances between best and worst useful to gauge estimate accuracy

  23. Work Break down Structure

  24. Risk categories Technical Test tool Application/product Communication

  25. Risk management

  26. PREDICT Model - Design Plan Rationalize Estimate Transition Design Consolidate Implement

  27. Test suite design issues Testability Test suite design patterns

  28. Testability considerations Tools Design Naming Conventions Exception & Error Handling Timing issues Logging Visual Cue Processes & Documentation

  29. Test automation development approaches • Record/Play back • Functional de-composition • Key-word Driven • User Interface-driven • Model-based

  30. Record & Playback • Advantages • Useful in determining how the tool interacts with the application under test • Provides initial ideas on how to develop test scripts • Useful while ‘Playing around’ with the tool • Disadvantages • Test scripts contains hard-coded values • Un-reliable scripts : Any small change in the application results in not being able to replay • Maintenance costs are astronomical • Not viable & cost-effective

  31. Functional decomposition • Reduces test scripts into their fundamental tasks • Driver Scripts : Initialization, Calling scripts in a desired order • Generic scripts : General,Application specific,Navigation,Data verification,Reporting,logging etc… • Business Function Scripts : ‘Re-usable’ Business functionality,Makes calls to Generic Scripts • Test Script : Test Logic, Makes calls to Business Function Scripts and Generic Scripts • Test data and logic are separated • Hierarchical structure is employed : Modular design • Each test script  unique data file

  32. Technical Architecture Stored Proc ObjectRepository EnvironmentConfiguration Test Director Generic Library BusinessLibrary Test Scripts Test Scripts Reports

  33. Functional decomposition: Analysis • Advantages • Modular design : Reduces redundancy and duplication of efforts • Test data input & output stored in a single data file : Easy maintenance • Scripts can be developed whilst application development is in progress : Generic Scripts • Single point maintenance is possible using Business Function scripts • Scripts can return TRUE (or) FALSE to the calling scripts. This makes it easy for ‘Error Recovery’ & un-attended test execution • Assemble tests on demand • Disadvantages • Proficiency in the TSL • Complex test data file management

  34. Key-word driven • The test script is represented as a set of key-words in a spread sheet • Examples of Key-words are start_test, Initial window,Url,Input,Verify etc.. • Driver Script Performs the initialization as required • Driver Script Reads and processes the file name(s) of the test scripts • Driver Script Matches the ‘Key Words’ contained in the file • Driver Script invokes Business Function & Generic scripts • Report the return values back • The entire process is data-driven

  35. Key-word driven - Analysis • Advantages • Preserves most of the advantages of the ‘Functional decomposition’ method • Very similar to documenting manual test script • Minimal programming knowledge is required to write and execute test scripts • Return on investment can be achieved quicker than other methods in most of the cases • Re-usability significantly increases over a period of time • Test script development is possible with out a functioning application • Disadvantages • Investment in initial frame work • Training might become tedious

  36. Technical Architecture GenericScripts GUI Map Keyword Driven Test Cluster Business Function Script1 Driver Script Business Function Script2 Business Function Script3 Reports Business Function Script4

  37. User interface driven • Abstraction of test structure: Test set  Test case  Test step  Test action • Key-words are identified based on the application under test • Test steps are created using user interface • Test set management (Creation/execution) Test set Test case Test step Test action

  38. Test Library Structure Execution Engine Automation Framework Recording Engine Script Generation Module Win Runner/QTP/Silk Test/… User Interface APPLICATION Test Script XML Object Repository Object Creation Module Technical Architecture

  39. User interface, Model driven - Analysis • Advantages • Preserves most of the advantages of the previous methods • Insulates tests both from the changes of the application and failings of test automation tools • Zero-programming required • Return-on-investment is very high • Re-usability significantly increases over a period of time • Robustness and maintainability is high • Partly application independent • Minimal learning curve • Challenges • High initial investment in creating framework • Functioning application needed

  40. Model Driven Behavior modeler Win Runner Test Case Generator (In key words) Keyword Translator engine (Test tool specific) Silk Test Test driver generator Other test tools Requirements (UML) Automated Test Scripts

  41. PREDICT Model - Implement Plan Rationalize Estimate Transition Design Consolidate Implement

  42. Test automation implementation issues Team structure Work allocation Coding guide-lines and check lists Peer reviews Test runs Integration Configuration management

  43. Work allocation

  44. Coding guidelines and checklist

  45. Reviews

  46. Building integrations Test Script1 Call Test Script1() Call Test Script2() Call Test Script3() Test Script2 Test Script3

  47. Configuration management : VSS

  48. Test execution Test suite execution check-lists are used Test Execution A separate test environment is prepared Fresh test data in test DB Fresh test data in Application DB Tests are executed Test suite execution issues & root cause analysis is documented Report Test Results Update test scripts if needed

  49. Test execution log

  50. PREDICT Model - Consolidate Plan Rationalize Estimate Transition Design Consolidate Implement

More Related