1 / 19

Automated Software Testing: Test Execution and Review

Automated Software Testing: Test Execution and Review. Amritha Muralidharan (axm16u). Contents. What is test execution? Phases of Testing Unit Testing Integration Testing System Testing Test Results Analysis of Regression Tests User Acceptance Testing

Download Presentation

Automated Software Testing: Test Execution and Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Automated Software Testing: Test Execution and Review Amritha Muralidharan (axm16u)

  2. Contents • What is test execution? • Phases of Testing • Unit Testing • Integration Testing • System Testing • Test Results Analysis of Regression Tests • User Acceptance Testing • Defect Tracking and Automated Tracking tool • Test Program Status Tracking and EVMS • Test Metrics Collection and Analysis • White Box • Black Box • Test Execution Review

  3. Test Execution Flow

  4. What Is Test Execution? • Stage 5 of the ATLM • Creation of testing environment • Tests are carried out on the application • Input = test procedures • Output = satisfied/altered acceptance criteria • Software Problem Reporting • E.g. CourseMarker

  5. Phases of Testing • Unit, Integration, System and User Acceptance testing. • Tests executed according to the Test Plan for each phase.

  6. Phases of Testing: Unit Testing • Consistent with Development Schedule • Automated Results Checkout Process • White Box Level • Involves detailed look at code • Best to execute unit tests as soon after the code has been created. • Static Analysis • Code Profiling • Development Test Tools • Test Documentation • Metric Collection • Process Improvement • Software development folders (SDF) and Unit Test Evaluation Criteria

  7. Phases of Testing: Integration Testing • “Units that are incrementally integrated are tested together based on control flow” • Some integration testing takes place during unit testing • Test procedures based on integration test design addressed earlier in the ATLM • Results are analyzed and documented • Refining Test Procedures • Some test scripts can be automated for re-use in system testing • End-user approval = end of unit and integration testing

  8. Phases of Testing: System Testing • Higher Level • Performed by a separate test team • May require many individual test procedures • Test Execution • False Negatives • False Positives • Software Problem Report

  9. Test Results/Analysis of Regression Tests • New application baseline • Build release notes should accompany the new build • Smoke Tests • Regression Testing (after defects have been fixed/modified) • Investigate high risk areas of code • Indicator of developer behaviour • Test team investigates areas with larger number of problem reports • Analysis is a clear indicator of whether test execution procedures are useful in locating errors. • Where more test effort is required • System testing is complete when criteria have been met

  10. User Acceptance Testing • Involves End-user participation • Consists of the subset of the suite of tests performed at system level • Defects at UAT level will be documented in SPRs • Satisfactory resolutions of level 1 and 2 SPRs = conclusion to testing

  11. Defect Tracking and New Build Process • Test team performs problem reporting operations in compliance with a defined process • SPR creation • E-mail Notification to configuration management(CM) and developers • New software is checked via CM tools • New software build issuance • Defect Priority • Fatal, high, medium, low • Engineering review Board (ERB) • Automated Defect Tracking Tool • Ensures reported defects receive proper attention • Central defect tracking repository • Has several basic characteristics • Automated Test Tools • Maintain test results/permit automatic generation of test results • Recommended that SPRs are generated per life-cycle stage

  12. Test Program Status Tracking and EVMS • Helps to efficiently handle problems that arise during test effort • Test engineer must produce meaningful reports based on test plan measures and metrics • Test log • Test coverage reports

  13. Test Program Status Tracking and EVMS Cont… • To effectively monitor testing progress and report to senior management, an earned value approach is used. • EVMS = one of the best ways to track progress • Tracks value of completed works and compares it to planned costs and actual • Provides true measure of schedule and cost status • Enables creation of effective corrective actions • 4 steps • Identify short tasks • Schedule each task • Assign a budget to each task • Measure progress of each task Earned value for work completed – planned budget = Schedule Variance Earned value for work completed – actual cost = Cost Variance

  14. Test Metrics Collection and Analysis • Metrics = key indicators of test coverage progress and quality of test effort • Separated into white box and black box testing efforts • HOWEVER, gathering too many metrics can be time consuming

  15. Test Metrics Collection and Analysis: White Box Testing • Targets application’s internal working • Measures the DEPTH of testing • Coverage Analysis • Code Analysis and Code Profiling e.g. Rational Quantify • Objective is to locate complex areas of source code • Fault Density (Fd) = Nd/KSLOC • Predicts the remaining number of faults • Design Complexity Metrics measures the number of ways a module can cal other modules

  16. Test Metrics Collection and Analysis: Black Box Testing • Focus on the BREADTH of testing • Based on application’s external considerations • Each metric is assigned to one of three categories • Coverage • Progress • Quality

  17. Test Metrics Collection and Analysis: Black Box Testing • Coverage • Test Coverage • System Coverage • Functional Coverage • Progress • Test Procedure Execution Status • Error Discovery Rate • Defect Aging • Defect Fix Rate • Defect Trend Analysis • Quality • Test Success Index • Defect Density • Test Effectiveness • Problem Report Acceptance Criteria Metric • Test Automation Metric

  18. Test Execution Review • Stage 6 of ATLM • Review test performance e.g. metrics • Helps determine enhancements to be made • Document well performed activities • Lessons Learned should be acknowledged/documented • Aim to ensure best practices are used • Integrate a continual iterative learning process • Encouraged as improvement opportunities • “check pulse” • Documentation should be easily accessible to all team members • Corrective actions suggested must be evaluated • Represents he need for process improvement

  19. Questions?

More Related