1 / 19

Informatics 43 – May 20 , 2014

Informatics 43 – May 20 , 2014. Quality Assurance. What software qualities do we want to assure? Correctness. Quality Assurance. What software qualities do we want to assure? Correctness How? Testing Inspections and reviews Proofs, formal methods Static analysis. Testing Challenges.

base
Download Presentation

Informatics 43 – May 20 , 2014

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Informatics 43 – May 20, 2014

  2. Quality Assurance • What software qualities do we want to assure? • Correctness

  3. Quality Assurance • What software qualities do we want to assure? • Correctness • How? • Testing • Inspections and reviews • Proofs, formal methods • Static analysis

  4. Testing Challenges EdsgerDijkstra: “Testing can show the presence of bugs, but not their absence.” True? Why?

  5. Testing Challenges EdsgerDijkstra: “Testing can show the presence of bugs, but not their absence.” True! Because… Too many input-output pairs (test-cases) to exhaustively verify, so we must take a small sample. Example: multiplier, inputs two integers, outputs one integer

  6. Testing Challenges Let’s be practical. Testing can only select a very small set of inputs. Our goal should be to choose the best ones. What are the best 5 test cases for a multiplier? (AKA: what 5 test cases, if they don’t reveal any bugs, will give you the most confidence that the multiplier is working correctly?)

  7. Result of running test cases -- 100% Confidence in module being tested Sweet spot? Number of test cases with correct outputs

  8. The landscape of “bugs” Error (in programmer’s mind) Fault or defect (in code) Failure (in execution or output)

  9. The landscape of “bugs” Error (in programmer’s mind) Fault or defect (in code) Failure (in execution or output)

  10. Goals for selecting test cases • Detect defects • Locate defects • Qualities • Accurate • Complete thorough • Repeatable • Systematic

  11. Levels and types of testing • Unit / component • Integration testing (at many levels; known as system testing when the entire codebase is involved) • Acceptance • Volume / stress • Regression

  12. Restatement of goals Want to verify software’s correctness  Need to test  Need to decide on test cases  No set of test cases is sufficient

  13. Restatement of goals Want to verify software’s correctness  Need to test  Need to decide on test cases  No set of test cases is sufficient What is a systematic approach to the selection of test cases that will lead to accurate, thorough, repeatable identification of bugs?

  14. Approaches to test case selection • “White box” – or structural • Test cases based on the structure of the code • Can be partially automated • “Thorough”ness measured against code qualities • “Black box” – or specification based • Test cases based on specifications • Requires “formal” specs to automate • “Thorough”ness measured against either • the set of functionality, or • the set of inputs

  15. White box example • What test cases are required • to make sure every line of code is executed at least once? • to make sure every branch is taken at least once? • to make sure that all possible exceptions are thrown? a 17 read b from console if a < b while a < b-3 a  a / (b–50) print a, b

  16. White box example • What test cases are required • to make sure every line of code is executed at least once? “Code coverage” • to make sure every branch is taken at least once? “Branch coverage” • to make sure that all possible exceptions are thrown? a 17 read b from console if a < b while a < b-3 a  a / (b–50) print a, b

  17. Black box example • From the Requirements Specification: • “A substitute can modify the maximum number of miles he or she is willing to travel to a school.” • What test cases are required to • make sure the user interface actually updates internal data stores? • determine if a modified value is correctly used when assigning the substitute to a school? • verify that invalid inputs are rejected?

  18. Black box test cases • “A substitute can modify the maximum distance he or she is willing to travel to a school.” • make sure the user interface actually updates internal data stores? • Input: Sub Maria Gomez, new max: 23, • Output: look at Sub Info Page • determine if a modified value is correctly used when assigning the substitute to a school? • verify that invalid inputs are rejected? • Inputs: TwentyThree, -10, 0

  19. The Testing Process Model • Decide what to test. • Select a test case input. • Determine the expected output E. • Run the system with the test case input. • Capture the actual output A. • Compare E and A. • Different? Inform programmer • Loop back to 1 or 2, if time permits.

More Related