1 / 25

Chapter 13

Chapter 13. Testing Strategies. Chapter 13 – Testing Strategies Overview. Software Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user. We will discuss: Strategic Issues of Software Testing

krista
Download Presentation

Chapter 13

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 13 Testing Strategies

  2. Chapter 13 – Testing Strategies Overview • Software Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user. • We will discuss: • Strategic Issues of Software Testing • Testing Strategy for Conventional Software • Testing Strategy for Object-Oriented Software • Validation Testing • System Testing • Debugging

  3. 13.1 A Strategic Approach to Software Testing • Verification vs. Validation • Verification: A set of activities that ensure that software correctly implements a specific function. (Are we building the product right?) • Validation: A set of activities that ensure that the software that has been built is traceable to customer requirements. (Are we building the right product?) • Questions: • What is an example of an error that “Verification” / “Validation” testing of VT’s blackboard might uncover. • Does one of these deserve more testing resources, i.e. is one more important than the other?

  4. 13.1 A Strategic Approach to Software Testing independent tester developer • Questions: • What are some benefits of having an independent test group? • What are some problems that might be associated with the creation of an independent test group?

  5. 13.1 A Strategic Approach to Software Testing 1. unit test 2. integration test 4. system test 3. validation test

  6. 13.1 A Strategic Approach to Software Testing Strategy: • We begin by ‘testing-in-the-small’ and move toward ‘testing-in-the-large”’: • For conventional software • The module (component) is our initial focus • Integration of modules follows • For OO software • our focus when “testing in the small” changes from an individual module (the conventional view) to an OO class that encompasses attributes and operations and implies communication and collaboration. Questions: • The book strongly suggests the small to large approach. Do you see any potential drawbacks or pitfalls to this approach?

  7. 13.2 Strategic Issues • Specify product requirements in a quantifiable manner long before testing commences. • State testing objectives explicitly. • Understand the users of the software and develop a profile for each user category. • Develop a testing plan that emphasizes “rapid cycle testing.” • Build “robust” software that is designed to test itself • Use effective formal technical reviews as a filter prior to testing • Conduct formal technical reviews to assess the test strategy and test cases themselves. • Develop a continuous improvement approach for the testing process.

  8. 13.3 Testing Strategy of Conventional Software module to be tested interface local data structures boundary conditions independent paths error handling paths test cases

  9. 13.3 Testing Strategy of Conventional Software Example Function (Pseudo Code) Function Calculate_Letter_Grade (Integer Numeric_Grade) { Letter_Grade = “” if numeric_grade >= 90 then Letter_Grade = “A” if numeric_grade >= 80 and < 90 then Letter_Grade = “B” if numeric_grade >= 70 and < 80 then Letter_Grade = “C” if numeric_grade >= 60 and < 70 then Letter_Grade = “D” return letter_grade }

  10. 13.3 Testing Strategy of Conventional Software • Example Unit Test (Pseudo Code) • Procedure Test_Letter_Grade_Conversion () • { • Grade = 89 • Expected = B • Actual = Call Calculate_Letter_Grade (89) • If “Actual” not equal “Expected” then “Test Failed” • If “Actual” equals “Expected” then “Test Passed” • } • Questions: • How would this unit test change if we wanted it to be a “boundary” test?

  11. Unit Test Environment driver interface local data structures boundary conditions Module independent paths error handling paths stub stub test cases RESULTS

  12. 13.3 Testing Strategy of Conventional Software (Integration Testing) • Options: • • the “big bang” approach • • an incremental construction strategy • Questions: • Are there any situations where the “big bang” approach might be effective?

  13. 13.3 Testing Strategy of Conventional Software (Top- Down Integration Testing) A top module is tested with stubs B F G stubs are replaced one at a time, "depth first" C as new modules are integrated, some subset of tests is re-run D E

  14. 13.3 Testing Strategy of Conventional Software (Top- Down Integration Testing) View Course Grades Verify Student ID Lookup Grades for Student Assignment 1 = 90 Assignment 2 = 85 Assignment 3 = 95 CalculateCourse Average Average = 90 Translate Number to Letter 90 = A-

  15. 13.3 Testing Strategy of Conventional Software (Top- Down Integration Testing) View Course Grades Verify Student ID Lookup Grades for Student Assignment 1 = 90 Assignment 2 = 85 Assignment 3 = 95 CalculateCourse Average Average = ( 90 + 85 + 85 ) / 3 Translate Number to Letter 90 = A-

  16. 13.3 Testing Strategy of Conventional Software (Top- Down Integration Testing) View Course Grades Verify Student ID Lookup Grades for Student Assignment 1 = 90 Assignment 2 = 85 Assignment 3 = 95 CalculateCourse Average Average = ( 90 + 85 + 85 ) / 3 Translate Number to Letter If Number > 90 then A If Number > 80 and < 90 then B If Number > 70 and < 80 then C If Number > 60 and < 70 then D

  17. 13.3 Testing Strategy of Conventional Software (Bottom-Up Integration Testing) A B F G drivers are replaced one at a time, "depth first" C worker modules are grouped into builds and integrated D E cluster

  18. 13.3 Testing Strategy of Conventional Software Regression Testing • The re-execution of some subset of tests that have already been conducted to ensure that changes don’t have unintended side-effects. • Important strategy for reducing “side-effects” Smoke Testing • Term originated from an “electrician’s” test • A rolling integration strategy where daily software builds are produced and then tested with surface level tests intended to find major defects. Questions: • What are some advantages/disadvantages to smoke testing?

  19. 13.4 Testing Strategy of Object Oriented Software • the concept of the ‘unit’ broadens due to encapsulation • class testing is the equivalent of unit testing • operations within the class are tested • the state behavior of the class is examined • integration applied three different strategies • thread-based testing—integrates the set of classes required to respond to one input or event • use-based testing—integrates the set of classes required to respond to one use case • cluster testing—integrates the set of classes required to demonstrate one collaboration

  20. 13.5 Validation Testing • Validation testing • Focus is on software requirements • Validation Test Criteria • Test plans- High level document describing the level of effort required, timeline, resouces, required, success measures, etc. • Test Procedure – Execution steps of a test. • Configuration Review • Also called an “audit” • Alpha/Beta testing • Focus is on customer usage. • Alpha Testing is on-site while Beta Testing is at the customer’s site. Questions: • What would provide a good basis for validation testing? • Who should perform the validation test? Developers or users? • Why is beta-testing such an effective means of software testing?

  21. 13.6 System Testing • Recovery testing • forces the software to fail in a variety of ways and verifies that recovery is properly performed • Security testing • verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration • Stress testing • executes a system in a manner that demands resources in abnormal quantity, frequency, or volume • Performance Testing • test the run-time performance of software within the context of an integrated system

  22. 13.7 The Art of Debugging test cases results new test cases regression tests suspected causes Debugging corrections identified causes

  23. 13.7 The Art of Debugging Symptoms & Causes symptom and cause may be geographically separated symptom may disappear when another problem is fixed cause may be due to a combination of non-errors cause may be due to a system or compiler error cause may be due to symptom assumptions that everyone cause believes symptom may be intermittent

  24. Consequences of Bugs infectious damage catastrophic extreme serious disturbing annoying mild Bug Type Bug Categories: function-related bugs, system-related bugs, data bugs, coding bugs, design bugs, documentation bugs, standards violations, etc.

  25. 13.7 The Art of Debugging Debugging Techniques: • Brute force - memory dumps and run-time traces are examined for clues to error causes • Backtracking - source code is examined by looking backwards from symptom to potential causes of errors • Cause elimination - uses binary partitioning to reduce the number of locations potential (where errors can exist)

More Related