1 / 24

Nightly Releases and Testing

Nightly Releases and Testing. Alexander Undrus. Atlas SW week, May 24 2004. Outline. Status of Nightly Builds New NICOS version 0.3 Testing in Nightly Builds. Status of Nightlies. Three nightlies: opt, dbg, “8.0.x” 2x2GHz machines: 6 hrs gain: Make is faster by factor 1.5 – 2.0

joshua
Download Presentation

Nightly Releases and Testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nightly Releases and Testing Alexander Undrus Atlas SW week, May 24 2004

  2. Outline • Status of Nightly Builds • New NICOS version 0.3 • Testing in Nightly Builds Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  3. Status of Nightlies • Three nightlies: opt, dbg, “8.0.x” • 2x2GHz machines: 6 hrs gain: • Make is faster by factor 1.5 – 2.0 • Builds are finished by ~ noon (CERN time) • More efforts for acceleration • Parallel checkout: saves 1 hour but connection problems with CVS server • Multithreaded builds: new tests with CMT v1r16 start soon • Incremental builds (Simon George) Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  4. Plan for better robustness of Nightlies • Builds on local disk followed by copy to AFS • Better stability, decreasing AFS load • Planned setting: • Release is checked out to local disk • Opt & dbg builds start simultaneously • Interlock mechanism for building NOVA • Copy to AFS (script copy_release available) • Waiting Database team for implementation of NOVA interlocks Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  5. Purpose of NICOS • NIghtly COntrol System • Provide flexible nightly build system for coordination of distributed development • Test new software submissions • Send immediate feedback to developers • Facilitate multi-person, multi-platform development of code • Portable System (perl based) Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  6. NICOS Organization Code Checkout NICOS Job NICOS Controller Project Setup Packages Database Project Make NICOS Webmaster Configur. Cache Testing Version Admin. Error Analysis Atlas SW week, Dec.03 - Alex Undrus - NICOS 7

  7. History of NICOS • Fall 2001: start of ATLAS nightly builds with set of shell scripts • Jan 2003: start of development of NICOS, perl based nightly control system • Feb 2003: version 0_1 of NICOS is released and applied for POOL nightly builds (LCG project) • July 2003: NICOS is accepted as LCG nightly build tool, used in POOL and SEAL builds • Nov 2004: NICOS version 0_2 applied for ATLAS builds Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  8. NICOS version 0.3 • Improved pages design: • Tag list emphasizes new packages and versions • Three sorting options for list of packages • Added category of QA tests (to previously used integrated and unit tests) • Test results on separate web page, while page with build results contain test summary • Added “warning” level for minor problems • Builds “on demand” (use NICOS in users work environment), tested by S. George Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  9. NICOS version 0.3 (cont.) • Build results available immediately after make loop • Integration with QMTest, Python based testing tool, supported by LCG Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  10. NICOS Project Web Page new Atlas SW week, Dec.03 - Alex Undrus - NICOS 12

  11. NICOS Tag List NEW NEW Atlas SW week, Dec. 2003 - Alex Undrus - NICOS 13

  12. NICOS Build Results NEW NEW Atlas SW week, Dec.03 - Alex Undrus - NICOS 14

  13. NICOS Test Results NEW CHEP’03 - Alexander Undrus - NICOS system 15

  14. Nightly Testing Principles • Automation • Smoke tests • Support different granularity levels • QA tests • Unit (structural) tests • Integrated (behavioral) tests • Use of test tools (CppUNIT, QMTest) • Convenience for developers • Fast feedback to users Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  15. QMTEST • Open-source, cross-platform testing tool • Python based, extensions possible • Tests are described in xml format • Organizes tests hierarchically (in suites) • Provides a graphical interface for creating and running tests • Also can be used from command line Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  16. QMTEST & NICOS integration • Integration is provided by Python script (qmtest_config): • Searches for test scripts • NICOS test database • Release tree (in directories test) • Determines test suites • NICOS test database • Pattern “TEST_SUITE” in script body • Tries to guess • Creates XML descriptions for QMTest Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  17. QMTEST & NICOS integration (cont.) • QMTest runs tests, determines results (regression is possible), handles log info • Results are reflected on NICOS pages and also can be browsed with QMTest graphical interface • Test can be rerun from QMTest graphical interface Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  18. NICOS & QMTEST integration NICOS Job qmtest_config.py QMTEST Test Database XML test descriptions NICOS Webmaster Atlas SW week, Dec.03 - Alex Undrus - NICOS 7

  19. QMTEST page CHEP’03 - Alexander Undrus - NICOS system 15

  20. QMTEST suite CHEP’03 - Alexander Undrus - NICOS system 15

  21. CppUnit • Tool for unit testing of C++ code/classes • allows to run tests automatically and gives a report about successes and failures • For developers that write software and test simultaneously • ATLAS TestPolicy provides access to LCG installation of CppUnit and integrates it in CMT environment (since January 2004). • Details: http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/nicoswww/cppunit_testing.html Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  22. Status of NICOS 0.3 • Trial 8.1.0 builds on lxbuild005 local disk: • Since ~ May 10 • atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/testww • Only one significant problem revealed: QMTest does not work with PyXML used by ATLAS • Temporary solution: exclusion of PyXML from PATHs • Test in real nightlies is scheduled on the next weekend • Update of manual to be completed by June 1 Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  23. To Do list for NICOS • Improve convenience features for “builds on demand” • Create scripts for administrative tasks for stable releases (rebuilds, tests, documentation) Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

  24. Further Information • ATLAS nightly builds web page atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/atlaswww/nightly_builds.html • NICOS web page www.usatlas.bnl.gov/computing/software/nicos/index.html • QMTest integration page http://atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/OO/dist/nightlies/nicoswww/qmtest_atlas.html Atlas SW week, May 24, 04 – Alex Undrus – Nightly Builds

More Related