1 / 10

Discussion on Testing

Discussion on Testing. Simantics Technical Board Meeting 30.1.2012. Organizing Test Suites. Suggestion: new plug-in org.simantics.tests Contains test suites for different testing activities: Regression TDD Performance Stress

bendek
Download Presentation

Discussion on Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discussion on Testing Simantics Technical Board Meeting 30.1.2012

  2. Organizing Test Suites • Suggestion: new plug-in org.simantics.tests • Contains test suites for different testing activities: • Regression • TDD • Performance • Stress • Composes test suites for the SDK from around the codebase, doesn’t contain test cases • Most suites are automatically tested via Jenkins

  3. Testing Activities • Regression • First step is to move all currently existing and passing tests into regression tests • TDD • Development-time, moved into regression when development is complete, manually executed • Performance/Scalability • Focus on performance-critical parts of the SDK right now: databoard, DB, graphics, history • Stress • For uncovering hidden unstabilities, mainly relevant for DB

  4. Tools to Incorporate • Squish (finally) • EMMA (code coverage for unit tests) • Possibly static analysis tools

  5. Static Analysis • Several (free) tools available for Java [1] that have Eclipse & Jenkins plug-ins : • PMD, FindBugs, Checkstyle • AFAIK, none of us has been actively using any of these with Simantics • Relatively cheap to automate • Need to test the tools first to say anything definitive of usefulness

  6. What to Test? • org.simantics.tests: everything that is part of the platform – not products built on top of it (sysdyn, modelica, etc.) • A separate test suite for every product • UI testing • Difficult without a Workbench product • Use movie tutorial ? • Perhaps create the planned diagram editor for it ?

  7. Reporting • Jenkins provides unit/squish test reports • Performance testing needs different logging • Instrument all tested code with necessary performance logging (See Eclipse tracing facility) • See about using Apache JMeter (Jenkins plug-in) • For stress tests it is important to have proper logging enabled and relevant data kept for failed tests (databases, etc.)

  8. Epics (prioritized) • Create and automate org.simantics.tests • Sysdyn product test suite with UI tests • List most important testing topics • Coverage/Static analysis tools into use • Performance testing plan • How to measure? Relativity of results? Reporting/tools? • Prevent publishing of builds with broken tests & ensure email is delivered about breakage • Try to force developers to react to acute problems

  9. Working with tests and failure • Regression tests divided into three suites: • 1st: only intended to contain tests that work. It is important to know when something breaks, and it is easiest to discover this when builds turn from green to red/yellow or vice versa. • 2nd: broken tests that have been moved from 1st if fixing the bug that caused the regression is not possible immediately. • 3rd: broken tests that have no incoming fix for a long time. 2nd->3rd moves done only at the turns of sprints.

  10. References • [1] http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis#Java

More Related