1 / 10

Discussion on Testing

Discussion on Testing. Simantics Technical Board Meeting 30.1.2012. Organizing Test Suites. Suggestion: new plug-in org.simantics.tests Contains test suites for different testing activities: Regression TDD Performance Stress

eagan-holden
Download Presentation

Discussion on Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discussion on Testing Simantics Technical Board Meeting 30.1.2012

  2. Organizing Test Suites • Suggestion: new plug-in org.simantics.tests • Contains test suites for different testing activities: • Regression • TDD • Performance • Stress • Composes test suites for the SDK from around the codebase, doesn’t contain test cases • Most suites are automatically tested via Jenkins

  3. Testing Activities • Regression • First step is to move all currently existing and passing tests into regression tests • TDD • Development-time, moved into regression when development is complete, manually executed • Performance/Scalability • Focus on performance-critical parts of the SDK right now: databoard, DB, graphics, history • Stress • For uncovering hidden unstabilities, mainly relevant for DB

  4. Tools to Incorporate • Squish (finally) • EMMA (code coverage for unit tests) • Possibly static analysis tools

  5. Static Analysis • Several (free) tools available for Java [1]that have Eclipse & Jenkins plug-ins : • PMD, FindBugs, Checkstyle • AFAIK, none of us has been actively using any of these with Simantics • Relatively cheap to automate • Need to test the tools first to say anything definitive of usefulness

  6. What to Test? • org.simantics.tests: everything that is part of the platform – not products built on top of it (sysdyn, modelica, etc.) • A separate test suite for every product • UI testing • Difficult without a Workbench product • Use movie tutorial ? • Perhaps create the planned diagram editor for it ?

  7. Static Analysis • Several (free) tools available for Java [1]that have Eclipse & Jenkins plug-ins : • PMD, FindBugs, Checkstyle • AFAIK, none of us has been actively using any of these with Simantics • Relatively cheap to automate • Need to test the tools first to say anything definitive of usefulness

  8. Reporting • Jenkins provides unit/squish test reports • Performance testing needs different logging • Instrument all tested code with necessary performance logging (See Eclipse tracing facility) • See about using Apache JMeter (Jenkins plug-in) • For stress tests it is important to have proper logging enabled and relevant data kept for failed tests (databases, etc.)

  9. Epics (prioritized) • Create and automate org.simantics.tests • Sysdyn product test suite with UI tests • List most important testing topics • Prevent publishing of builds with broken tests & ensure email is delivered about breakage • Try to force developers to react to acute problems

  10. References • [1] http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis#Java

More Related