CS 49000-ST0 Software Testing Spring 2011 Review

1. CS 49000-ST0 Software TestingSpring 2011Review Aditya P. Mathur Purdue University Last updated: April 26, 2011

2. Learning Objectives • Various test processes and continuous quality improvement • Types of errors and fault models • Methods of test generation from requirements • Behavior modeling using UML: Finite state machines (FSM) • Test generation from FSM models

3. Learning Objectives (contd) • Input space modeling using combinatorial designs • Combinatorial test generation • Test adequacy assessment using: control flow, data flow, and program mutations • The use of various test tools • Application of software testing techniques in commercial environments

4. Test processes and continuous quality improvement Test plan: Chapter 1.5 Test metrics: Chapter 1.6 Defect management: Chapter 1.9 Walkthroughs and inspection: Chapter 1.12 Test process models: Chapter 1.18.5

5. Types of errors and fault models Chapter 1.1

6. Methods of test generation from requirements Equivalence partitioning: Chapter 2.3 Boundary value analysis: Chapter 2.4

7. Behavior modeling using UML: Finite state machines (FSM) FSM basics: Chapter 3.2 Microsoft project AEC project: parsing

8. Test generation from FSM models Spec Explorer W-method: Chapter 3.6 State, transition, and switch covers: Chapter 3.9

9. Input space modeling using combinatorial designs Test configurations and modeling the input space: Chapter 4.1 Fault model: Chapter 4.3

10. Combinatorial test generation MOLS: 4.8 Covering arrays: 4.9, 4.10, 4.11

11. Test adequacy assessment using: control flow, data flow, and program mutations Basics: 6.1 Criteria based on control flow: 6.2 (excluded 6.2.7) Criteria based on data flow: 6.3, 6.4.1, 6.4.2, 6.4.3

12. The use of various test tools Unit testing: Achoo,JUnit, Pex/Moles, PyUnit Concurrency: ConTest Test automation: TestComplete Load and performance testing: JMeter GUI testing: Abbot, Guitar Combinatorial testing: AETG, FireEye Model based testing testing: Spec Explorer Coverage analysis: Corbertura Defect management: Bugzilla

13. Application of software testing techniques in commercial environments Some Spec Explorer Statistics [MS]: Team Size test cases
600 798 
158 8178 
100 100 
60 164 
40 40 
30 900 
20 629 
18 3000 
13 1900 
10 150 
10 60 
8 28 AEC project Microsoft project Some Spec Explorer Statistics [Customers]: Team Size test cases
20 1282
12 250


14. What did we not cover [1]? Regression testing (Chapter 5) Mutation testing (for test adequacy and enhancement; Chapter 7) Test generation from statecharts Test generation from timed-automata for real-time systems Test generation from formal specification (e.g.; Z) Testing concurrent programs

15. What did we not cover [2]? Test process Test automation Error types and classification method

16. Final Examination Tuesday May 3. 8-10am. Smith 208 Part A: Multiple choice, comprehensive [Closed book/notes] Part B: Few questions (3 or 4) [Open book/notes] FSM-based test generation, Combinatorial designs, Test adequacy

17. Good luck! Best wishes!