1 / 10

Pipeline Test methods

Pipeline Test methods. Lars Kr. Lundin - CPL developer - NACO and VISIR developer - SPHERE Contact Person. Presentation Outline. Static code analysis Unit test Integration test Regression test Testing via nightly pipeline builds. Static Code Analysis. Static Code Analysis:

whitby
Download Presentation

Pipeline Test methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pipeline Test methods Lars Kr. Lundin - CPL developer - NACO and VISIR developer - SPHERE Contact Person

  2. Presentation Outline • Static code analysis • Unit test • Integration test • Regression test • Testing via nightly pipeline builds

  3. Static Code Analysis Static Code Analysis: The analysis of the source code without program execution. Find programming errors and non-conforming code. Tools: • gcc: Various warning-options (-Wall, -Wextra, …) • splint • Other C-compilers (e.g. Solaris)

  4. Unit Test Unit test: Validate the individual units (i.e. functions) of the source code. Each unit test should be implemented before the functionality it is validating: Its initial failure will validate its testing ability. Benefits: • Exemplifies the intended usage of the tested function • Reduces complexity of integration testing • Errors in later changes will be exposed (regression bugs) • Unit testing can be automated

  5. Unit test - build system iiinstrument/tests/Makefile.am : check_PROGRAMS = iiinstrument_my1st_module-test \ iiinstrument_my2nd_module-test iiinstrument_my1st_module_test_SOURCES = \ iiinstrument_my1st_module-test.c iiinstrument_my2nd_module_test_SOURCES = \ iiinstrument_my2nd_module-test.c TESTS = iiinstrument_my1st_module-test \ iiinstrument_my2nd_module-test

  6. Unit test – code example iiinstrument/tests/iinstrument_my1st_module-test.c: int main (void) { /* Test error handling and normal call */ assert( iiinstrument_my1st_function(NULL) == CPL_ERROR_NULL_INPUT); assert( iiinstrument_my1st_function(“OK”) == CPL_ERROR_NONE); return 0; }

  7. Integration Test Integration test: Test the combination of individual software components as a group, i.e. a complete pipeline recipe with a front-end, i.e. esorex and gasgano. Involves (manual) validation of the science products. Science validation is time consuming, thus done only after all automated tests are successful. Creates reference set of validated pipeline output data for subsequent regression testing.

  8. Regression Test Regression test: Uncover regression bugs, i.e. when functionality that used to work as required stops doing so. Regression bugs are typically introduced as an unintended consequence of a software change, possibly in a support library. Regression testing becomes essential once the development proceeds beyond the first working version.

  9. Regression Test Continued Partial automation of regression tests using the esorex regression test tool, esoreg.pl: • Execution of multiple tests (during lunch, night/week-end). • Detection of crashes, failures, missing products, memory-errors (valgrind). • Find differences in product-headers. • - report only largest change in a numerical FITS card. • Find largest change in every data unit. • Filter out insignificant changes (DATE related keys).

  10. Nightly Pipeline Builds Nightly Pipeline Builds: • Tests run every night using latest CPL release • Unit tests • Simple integration test with esorex • Various static checks (splint, staticcheck) • Memory errors in Unit and Integration tests (valgrind) • Multi-platform tests (Linux, Solaris, MacOS, HP-UX) • Multi-compiler tests (new and older gcc, Solaris/HP-UX cc) • http://www.eso.org/~llundin/cpl/qc/

More Related