Testing Computer Software 2 nd Edition (1999)

1. Testing Computer Software 2nd Edition(1999) CemKaner Jack Falk Hung Quoc Nguyen

2. Testing Computer Software • Objectives and Limits of Testing • Realistic test planning is dominated by the need to select a few test cases from a huge set of possibilities. • You cannot fully/completely test a program • The prime benefit of testing is improved quality

3. Testing Computer Software • Software Errors • User Interface Errors • Error Handling (one of the least tested areas) • Boundary-Related Errors • Calculation Errors • Initial and Later Stages (happens only the first time) • Control Flow Errors (menus to submenus) • Errors in Handling and Interpreting Data (between modules) • Race Conditions

4. Testing Computer Software • Software Errors (continued) • Load Conditions • Hardware • Source and Version Control • Documentation (manual, book) • Testing Errors (code, procedures, user error)

5. Testing Computer Software • Reporting and Analyzing Bugs • The point is to get bugs fixed • A good report is written, simple, understandable, reproducible, legible, and non-judgmental • The Bug Tracking System • The system introduces project accountability • Users of the system (Testers, Project Manager, Programmer, Product Manager, Technical Support, Writer, Test Manager, Senior Executives, Lawyers)

6. Testing Computer Software • The Bug Tracking System (continued) • The authors strongly discourage providing personal performance data • Generate many useful reports (weekly status, deferred, patches)

7. Testing Computer Software • Test Types and Test Case Design • Pre-testing stage • Requirements review • Design review • Coding stage • Whitebox/Glassbox Testing • Testing stage • Blackbox Testing • Functional Testing • System Testing • Load Testing • Configuration Testing • Test only one device per group to eliminate the errors common to that group. • User Manuals

8. Testing Computer Software • Test Types and Test Case Design (continued) • Equivalence Classes (same result from two different tests) • Functional Equivalence Testing (automation, random input) • Boundary Analysis • Testing state transitions (ex. Menus) • Race Conditions

9. Testing Computer Software • Test Types and Test Case Design (continued) • Regression (bug fixes, library of tests) • Testing Tools • Basic Tools • File compare Utility, File Viewers • Automated Regression Tests • Standards compliance-checking programs • Tools for code coverage, assertion checks, memory

10. Testing Computer Software • Test Planning and Documentation • Test Plan as a product and Test Plan as tool • Goal of Test Plans and Docs • Facilitate technical tasks of testing (study program, more complete coverage) • Describe testing tasks and processes (size, depth of testing job) • Organize testing project (improve individual accountability, schedule)

11. Testing Computer Software • Test Planning and Documentation (continued) • White box vs Black box • They are complementary. They find different types of bugs. • Suggest evolutionary method for developing test plan • Start small and build and add new sections as you go along. Avoid huge test plan at the start of testing.

12. Testing Computer Software • Test Planning and Documentation (continued) • Test Plan (defined by IEEE Standard 829) • Acceptance Test Criteria • Test Design spec • Test Summary Report (complete a cycle of testing) • Some Minor docs • Test case spec and Test procedure spec • Test item transmittal report • Test script • Test log

13. Testing Computer Software • Software Development Model implications for testing • Waterfall • You must review the UI early • Start writing test plan as early as possible (risk assess) • You cannot start testing until late in the project • When you start testing, you are on the critical path

14. Testing Computer Software • Software Development Model implications for testing • Evolutionary • Start reliability testing as soon as first level of functionality – plan to staff early • Plan waves of usability tests as project grows more complex • Write test plan as you go • Do your most powerful testing as soon as possible because project might be stopped and shipped at any time

15. Testing Computer Software • Managing a testing group • Management is the real quality control group in any company • Provide predictability to the project manager • Different Types of Testing groups • QC – enforces standards • QA – senior people involves at all stages • Testing services – find bugs • Development Services – debug, initial tech support