Download
1 / 12
120 likes | 315 Views
Improving Testability. Dave Catlett Test Architect, Microsoft davidcat@microsoft.com. Agenda. What is testability? Why is testability important? Background: What is a test? Determining testability: SOCK Improving testability Q&A. What is testability?.
E N D
Improving Testability Dave Catlett Test Architect, Microsoft davidcat@microsoft.com
Agenda • What is testability? • Why is testability important? • Background: What is a test? • Determining testability: SOCK • Improving testability • Q&A
What is testability? • The degree to which components and systems are designed and implemented to make it easier for tests to achieve complete and repeatable code path coverage and simulate all usage situations in a cost efficient manner. • In other words: How easy is it to test?
Why is testability important? • Helps enforce good design principles. • Reduces the time to diagnose unexpected behavior. • Improves manageability and supportability. • Provides the building blocks for self-diagnosing and self-correcting software. • Speeds test development while increasing the depth and quality of automated tests.
Background: What is a test? • Making sure the component works as expected and has the expected effect on the system as a whole in a repeatable and reliable way. • Anatomy of a test case: SEARCH • Setup • Execution • Analysis • Reporting • Cleanup • Help
Determining testability: SOCK • Simplicity • Reducing cost of testing at each phase of SEARCH • Design • Observability • Exposing state (aka Visibility) • Control • Exercising every nook & cranny as efficiently as possible. • Knowledge of expected results • Is observed behavior is correct?
Improving Simplicity • Make setup and configuration easy • Componentize • Allow multiple installs with varying configs on same machine • Use good design principles • Strong cohesion • Loose coupling • Low redundancy
Improving Observability • Expose component state • Magic numbers (Any value that controls code path execution.) • Resource usage • State transition information • API in/out parameters • Tracing • Good tracing library • Observer or Visitor patterns can help maintain good design • Be careful of exposing security info!
Improving Control • Wrap interfaces you have no control over • Forces behavior you need to test your component • Facilitates fault injection tools • Windows: Holodeck, AppVerifier • Expose Magic Numbers to become configurable at runtime. • Be mindful of security
Improving Knowledge of expected results • Code contracts in source code • Document expected behavior in all situations • Document end-to-end user scenarios • Future: Use modeling (machine readable specifications)
Resources • Design Patterns Explained: A new perspective on object-oriented design. Alan Shalloway, James Trott. Addison Wesley, 2002. • Observer pattern (tracing option for new classes) • Visitor pattern (tracing option for existing classes) • Adapter pattern (wrapping for fault injection) • Refactoring: Improving the design of existing code. Martin Fowler, et al. Addison-Wesley, 1999. • Refactoring to improve design helps testability at the same time. • www.netobjectives.com • Emergent Design, other materials • Fault injection tools for Windows • Holodeck: http://www.sisecure.com fault injection tool. • AppVerifier: http://www.microsoft.com/technet/prodtechnol/winxppro/deploy/appvrfr.mspx • Google: CCCC cyclomatic complexity tool • Brett Pettichord, Industry Testability Guru • Design for Testability: http://www.io.com/~wazmo/papers/design_for_testability_PNSQC.pdf • QA-related papers: http://www.io.com/~wazmo/qa • Other: http://www.pettichord.com
