1 / 40

Advancing Software Quality Testing Tools: A Comprehensive Overview

Explore the market demands and pressures for software testing in the 90s, new measures of success, quality assurance processes, use of evaluation tools, and advantages of test automation. The text highlights the benefits of test execution tools, automated testing processes, test case/data generation, and test management utilities.

guyjimenez
Download Presentation

Advancing Software Quality Testing Tools: A Comprehensive Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Research, Inc. Setting the Standard for Software Testing A Survey of Technology

  2. Pressures of the 90’s Marketplace • Customer changing their requirements • Customers demanding fewer bugs • Financial managers demanding lower costs • Managers shortening schedules • Developers wanting professional recognition • Competitors wanting your market share

  3. New Measures of SuccessGoing beyond simply meeting schedules • Customer reported defects • Cycle time on maintenance releases • Life cycle costs • Supplier reputation

  4. Ensuring Quality • Deliver tested code • Build quality — don’t test it in • Include testing and QA as part of product development

  5. Quality Assurance Process • Tool coordination/usage • QA and testing • Process and measurement

  6. Why Use Quality Evaluation Tools?It’s the most cost-effective way to verify that software and tests are high quality Evaluation Method Risk High defect rate Let customer check quality Low customer satisfaction High labor cost Check quality manually Error prone Cost of tool Check quality with tools Setup time

  7. Manual vs. Automated TestingTypical Run/Record Test Times 60 Minutes 30 30 3 Substantial time savings on reruns! 1st Test Reruns

  8. Benefits of Test Execution Tools • Tools save time and money because • They run test cases faster than people • They rerun test cases more reliably than people • They can rerun test cases unattended • They run tests for each release providing improved productTools can report difference between previously correct responses and current product behavior

  9. Test Automation • Test execution tools automate the repeatable test cases • Capture / Playback • True-time, Optical character recognition, widget (object oriented) • More valuable as the product stabilizes

  10. Test Automation . . . • Test Case/Data Generation • Automates the creation of test data • Driven by requirements/specifications • Random/sequential value selections • Saves substantial creation time • Improves overall test coverage

  11. Test Automation . . . • Test harness • Test independent subsystems • Creating an expected environment for the subsystem • Initiates automation at the unit test level

  12. The SolutionAn automated process that is . . . • Measurable with pin-point accuracy • Flexible and portable across environments • User friendly • Cuts software development time • Generates cost savings

  13. TestWorks Integrated Tool Suite

  14. STW/Regression • Test capture and playback • Test management facilities • Differencing utility • Language independent

  15. STW/RegressionMultiple record/playback modes for: • True Time • Optical Character Recognition • Object Oriented (Widget)

  16. Test Management • Organization of test tree • Automatic execution of tests • Automatic recording of results

  17. Test Coverage Used at unit, integration and system test levels • Line (unit) • Branch/Segment (unit, system) • Call-Pair (integration, system) • Path (unit, system)

  18. Coverage Benefits • Measures the completeness of test cases • Improves quality by focusing the creation of additional tests • Saves time by not creating tests for code that is already exercised • Improves process by providing measurements • Finds unexecutedcode (dead code/trap doors)

  19. Coverage Use • Developers’ unit test (Branch) • Test/QA during integration test (Call-Pair) • Test/QA during system test (Branch/CallPair) • Mission-critical module testing (Path Analysis)

  20. Coverage Features • Ease of integration with current build facilities • Multiple languages supported • Control over what code is instrumented

  21. More Coverage Features • Report Styles • Textual, Graphical • Reports • Hit, not hit, cumulative, charts, annotated listing • Language-specific features such as tasking and exceptions

  22. STW/AdvisorStatic • Unit and system test levels • Metrics (system test) • Give measurements to provide insight • Help focus efforts on suspect code • Help measure changes between projects

  23. STW/Advisor - Static con’t. • Syntax/Semantic analysis (unit test) • Highlights code that may be error-prone • Highlights code that may not be portable • Highlights code that does not meet coding standards

  24. Static Characteristics • Works with the entire program, not just files • Provides a wide variety of syntax and semantic analysis • Works with the languages you need • Is highly configurable to show only errors or critical warning signs

  25. STW/Advisor for Metrics • Halstead Software Science • Cyclomatic complexity • Support for: • Wide range of languages • Textual and graphical reports • Single files or entire programs • Settable ranges for acceptable results

  26. Load Generation • Regression • Generate realistic load and confirm behavior • Use multiple clients with people driving each one with centralized software/hardware • Use single workstation to emulate multiple sessions and protocol generation • Measure execution validation and timing

  27. Regression Same technology is used as host-based testing Extremely well suited for Client Server environment Tests GUI Ability to simulate multiple users on a single client Coverage Measures Client/ Server interaction Client Server

  28. Embedded Regression • Host and embedded system are connected through a test driver • Start the Application Under Test • Feed regression input to embedded system • Analyze response on the host system

  29. Embedded Coverage • Pre-process/instrument your code • Cross compile your code • Link runtime libraries • Upload Trace File from embedded system • Analyze results on host machine

  30. Control Capabilities • Scripts are in plain text; use standard editors • Timing mechanisms • True Time recording • Accelerated real time playback • Synchronization mechanisms • Automatic output sync • Window image sync • Partial image sync

  31. What To Look For In Automated Testing Tools • Cost of the tool • Training time required • Installation process • Technical support provided • How well the tool fits your requirements

  32. Important Characteristics • Ease of use • Improved testing capabilities • Cost amortization

  33. The Complete Solution TestWorks provides the best code coverage • Capture/playback for automation(STW/Regression) • Coverage analysis for effectiveness(STW/Coverage) • Metric/Static analysis for program mgmt. (STW/Advisor) • All products are fully integrated to provide the complete testing solution • Best feature/price ratio

  34. Software Research’s Unique Advantages • Fully integrated product line • Flexible tools approach • Products for both UNIX and Windows • Test management utility • The complete suite of testing tools in one package

  35. Multiple Platforms and Languages • UNIX-SUN, IBM, HP, DEC, SGI, and many others. • Windows NT/2000 and Windows XP • Languages: C, C++, ADA and others

  36. Flexible Tools Approach • GUI Interface for new users • Command-line equivalents for automated scripts and power users • Extensible-user contributed functions and access to shell services • Integration into user’s environment

  37. Software Research, Inc.Experts in Software System Testing 1977 Founded as a consulting and R&D firm 1987 First products released 1991 Released Software TestWorks, an integrated testing system 1995 Over 5,000 sites using TestWorks

  38. Customer Reviews • “SR won out because of functionality and price. It was the most effective of all the software that was researched.”-Jim Williams, U.S. Computer Services, Data Management Review

  39. Customer Reviews cont. • “Using the coverge tools I found code that I thought was being executed, but was not. With that information I was able to rapidly fix a bug that had been eluding me.-Bob Carasik, Pacific Bell • “We use Software TestWorks to drive our test process, providing a seamless solution through the analysis, design, test, and update/maintenance phases of our test effort. This saves us time while ensuring a high quality product is delivered.”-Charles Cooper, Manager, Quality Assurance, Ikos System, Inc.

  40. Software TestWorks SuiteThe Total Solution • Regression— We have it. • Coverage — We have it. • Advisor — We have it. • Full integration -- We have it. • One package. One vendor. One decision

More Related