1 / 37

Software Testing

http://flic.kr/p/7iffKj. Software Testing. SWEBOK Knowledge Areas. Software Requirements Software Design Software Construction Software Testing Software Maintenance Software Configuration Management Software Engineering Management Software Engineering Process

kaler
Download Presentation

Software Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. http://flic.kr/p/7iffKj Software Testing

  2. SWEBOK Knowledge Areas • Software Requirements • Software Design • Software Construction • Software Testing • Software Maintenance • Software Configuration Management • Software Engineering Management • Software Engineering Process • Software Engineering Models and Methods • Software Quality • Software Engineering Professional Practice • Software Engineering Economics • Computing Foundations • Mathematical Foundations • Engineering Foundations Today’s topic

  3. Can you create “real” software that is bug free? Alas, no Our goal: Reduce number of bugs – especially those users are likely to counter What is dominant means to discover bugs? Software testing

  4. “V & V” Activities • Verification: Check that system complies with requirements (based on your understanding) • Answers: Did we build the system right? • Validation: Check that system actually meets customer’s needs • Answers: Did we build the right system? Testing can be used for both

  5. Bug-Related Terminology • Defect/Bug/Fault: Thing that developer did wrong • Cause of program’s incorrect behavior • Error: Incorrect behavior of program • May not be observable to user • Failure: User-observable error; loss of functionality Some authors prefer to flip the above defs of fault and error

  6. Testing-Related Terminology • Testing: Execute code to reveal defects • Does it behave as expected? • Test/Test Case: One execution of code that may expose bug • Test Suite: Set of test cases • Often used to group related tests

  7. How do you completely test a program? Try all possible inputs—called exhaustive testing Can we do exhaustive testing in practice? No. Consider this example…

  8. Example: File operation • Takes file name as argument • Each character an 8-bit byte • How many tests cases to exhaustively test? If you limit file name to 10 characters,there are 25610 different test cases Even if you can run 1000 tests per second,it would still take over 1013 years! Clearly, exhaustive testing not feasible…

  9. The Testing Problem:How to choose small set of test casesthat reveal all errors? • Fundamental research problem • Essentially unsolvable in general case

  10. Decisions when designing tests • What subset of system to test? • Target of test • How to choose test cases?

  11. Decisions when designing tests • What subset of system to test? • Target of test • How to choose test cases?

  12. What subset of system to test? smaller subset Unit testing: Test modules in isolation; smallest “units” of code Integration testing: Test groups of collaborating units System testing: Test the complete system larger subset

  13. Unit versus Integration Testing • Problem: Definition of “unit” varies (class? method?...) • How to distinguish:

  14. Decisions when designing tests • What subset of system to test? • Target of test • How to choose test cases?

  15. How to choose test cases? • Blackbox testing: Choose based on module’s possible inputs and outputs • Do not use code • Often test boundary cases • Whitebox testing: Uses internal logic to choose tests • Different levels of code coverage • Aka glass box testing, clear box testing • Regression testing: Keep tests that reveal old bugs • Rationale: “Fixed” bugs come back!

  16. Many higher-level testing strategies as well • Acceptance testing: Users test to see if system meets actual use requirements • Serves both validation and verification • Usability testing: Observe users using system with eye on usability of system • Test-driven development: … more to come … • And many more…

  17. Tools are helpful and importantWhat affordances might testing tools provide?

  18. Tools are helpful and importantWhat affordances might testing tools provide? • Simplify writing test cases (e.g., with library) • Automatically execute tests • Summarize results of tests • Help “jump to” location of failure • Tell how “good” your test cases are (e.g., metrics) • Generate test cases for you

  19. Tools are helpful and importantWhat affordances might testing tools provide? • Simplify writing test cases (e.g., with library) • Automatically execute tests • Summarize results of tests • Help “jump to” location of failure • Tell how “good” your test cases are (e.g., metrics) • Generate test cases for you Particularly common in testing frameworks (e.g., JUnit, RSpec)

  20. Why is automated testing important?

  21. From a testing perspective, why might this scenario end badly? • Initial Plan: Implement software, then test at end • Implementation takes 6 months longer than expected • System is huge! • Finally, it’s time to test • System is huge, so testing job is HUGE • Pressure to ship causes skimping on testing • Defects discovered this late are costly to fix

  22. Defect Cost Increase (DCI) Principle The later you discover defect, the more expensive to fix Bad way: Time Better way: How to achieve? From Extreme Programming Explained by Beck & Andres

  23. Test-Driven Development (TDD) • Idea: Test First! • Write unit tests before functional code • Typically blackbox tests

  24. Running Example:Preorder Coffee with Gift Card

  25. US and Tasks Start withTask 1

  26. Further Break Down the Task • Represent order info • Represent gift card info • Represent receipt info

  27. Next Step: Write Test Code

  28. Rule #1: Your test should fail before you implement any code • Establishes a measure of success • Promotes programming incrementally

  29. Next: Write code to make the test pass

  30. Rule #2: Implement the simplest code possible to make the test pass • Also helps to promote incremental programming • By focusing on small bits of code • Helps resist urge to add unwanted extras • “Gold Plating” • Speculative generality: Adding lots of features to make reusable something that will never be reused

  31. Test-Driven Development Cycle

  32. Next (Red): Test order information

  33. In writing the test, you design the class interface — just enough interface! 1. Create an object 2. Set its attributes 3. Get and check the attribute values

  34. Next (Green): Implement the interface you designed for the test Instance variables “Setter” methods “Getter” methods

  35. If in the process of building up classes, you realize your design could be improved, thenREFACTOR!… and continue going around and around …

  36. As you go, you expand upon the systems capabilities • You might do a test for each of these cases: • A gift card with more than enough to cover the cost of the order • A gift card without enough to cover the cost of the order • An invalid gift card number • A gift card with exactly the right amount • A gift card that hasn’t been activated • A gift card that’s expired

  37. Pros/Cons of TDD • Pros: • Yields lots of test cases • More tests leads to increased confidence • Cons: • False sense of confidence? • Non-TDD folks may not understand why writing so many tests and not functionality Despite cons, TDD is a widely advocated practice

More Related