1 / 16

Test and Verification Solutions

Improved time to market through automated software testing Mike Bartley, TVS. Test and Verification Solutions. Agenda. The potential benefits & drawbacks of automation Compare with HW verif Can automation reduce TTM? Automating static & dynamic testing Deciding what to automate

ronni
Download Presentation

Test and Verification Solutions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improved time to market through automated software testing Mike Bartley, TVS Test and Verification Solutions

  2. Agenda • The potential benefits & drawbacks of automation • Compare with HW verif • Can automation reduce TTM? • Automating static & dynamic testing • Deciding what to automate • Choosing your automation tools • Case study

  3. The potential benefits of automation • Reduced costs • Improved quality • Improved employee motivation • Improved TTM

  4. The potential drawbacks of automation • Shelf-ware! • Maintenance • Skills base • Missing failures

  5. HW verification The mechanics of complex test bench Test Scoreboard Functional Coverage Coverage constraint addr data Monitor Design Under Test Assertions Stimulusgenerator Driver Coverage assert

  6. The mechanics of finding a bug Stimulate …..01010101 Propagate Design Under Test 01100101….. …..01001101 11110101….. …..10011010 00010101….. …..01001101 Observe Actual Results Expected Results Compare

  7. Can automation reduce TTM? • Getting test results sooner • Catching bugs earlier • Stabilising the software sooner • Assessing the stability of the software

  8. Automating static testing • Types of bug found • A quick win • Beware false failures • Introduce filters slowly • Developers not testers (check in checks, IDE) • No testing of intent

  9. Automating dynamic testing • Test plan • Test design specification • Test case specification • Test procedure specification • Test execution • Test log • Test incident report • Test summary report

  10. Deciding what to automate • Complexity of the test environment • The test level • Frequency of execution • Ease of automating pass/fail • Test stability and repeatability

  11. Choosing your automation tools • Who writes test test? • Matching skill set • Separate orthogonal concerns • Portability • Ease of adding/banning tests • Integration into configuration/build system • Cost of the tool?

  12. Case Study – Background • Includes chips, accelerator cards, and software • All work with industry-standard x86-based systems

  13. Case Study – Background • Low level host driver code • Code on host computer communicates with the chip on the board in the PCI slot • Libraries of code • Runtime library • Mathematics acceleration libraries (ACML, MKL, …) • Application code • Linpack, Montecarlo, …. • Code running on the chip • This had parallel C extensions not supported by our chosen tool • Software models of the hardware • In-house compiler for the code running on the chip • Mainly C and C++, some assembler

  14. Case study – automation details • Integrated into configuration/build system • Automated mostly lower level tests • Developed internally by SW Engineers • Offshored manual tests • Offshored some automated test development • Integration of static analysis • Test maintenance

  15. Case study - results of automation • Improved quality • Shortened release schedules • Release schedules hit • Reduced TTM • ROI?

  16. Questions? • Mike Bartley • mike@tandvsolns.co.uk • LinkedIn • 07796 307958

More Related