1 / 22

AliRoot and AliEn B uild I ntegration and T esting S ystem (BITS)

Automate the building of AliEn and AliRoot components, track dependencies, perform unit and functional tests, and provide a web interface for monitoring build status and accessing logs.

whanna
Download Presentation

AliRoot and AliEn B uild I ntegration and T esting S ystem (BITS)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AliRoot and AliEnBuild Integration and Testing System (BITS) Alina Grigoras alina.gabriela.grigoras@cern.ch Olga Vladimirovna Datskova olga.vladimirovna.datskova@cern.ch

  2. Having a current, up to date, installable binary image of AliEn and AliRoot for different architectures • Automate the process of building AliEn and AliRoot components • Introduce a better understanding of the dependencies between packages • Perform unit and functional tests during build time, deploying error alerts immediately • Provide a web interface that allows monitoring of the current build status and remote access to logs BITS purpose

  3. AliEn and AliRoot are supported for the following platforms: • SLC5 32 and 64 bits • SLC5 Itanium • IntelMac 32 and 64 bits Leopard • IntelMac 32 and 64 bits Snow Leopard (to be added) • Ubuntu 9.10 64 bits (testing purpose) Supported Platforms • AliEnBITS Web Interface Main Menu

  4. Based on the Konstruct framework used to build KDE and dependencies which in itself is based on BSD ports system • Each package is defined as a directory in a hierarchy • For each package there is a Makefile containing a set of variables: • Name, version, author, web page • License information • Build and runtime dependencies • Master sites for downloading the package source • Additional patches needed for AliEn/AliRoot environment • Checksums for verifying the integrity of the downloaded sources Build environment overview

  5. Get the available Releases • Select the one to be (re)built • Fetch the latest changes for this release • Do a cvs update • Determine the build order • Analise the dependencies between packages • Select the packages that have to be rebuilt • Based on modifications received on cvs update • Based on last build status • Based on package dependencies Build process[1]

  6. For each package • Clean its environment • Fetch source archive from master sites • Build, install and run tests • Create the binary image for binary installations • Generate package web page • Build / SLOC / Graph dependencies • Cleanup environment • Send notification email when status changes Build process [2]

  7. AliEn: packages graph

  8. http://alienbuild.cern.ch:8888 • The current build system maintains the following versions of AliEn across all supported platforms: • The build also provides the following AliEn distributions (the sizes are for v2-18 SLC5 64 bits): AliEn: available builds

  9. AliEn: Release Web Page

  10. AliEn Package Details - general information about the package - links to the Makefile + the applied patches Package Web Page AliEn Package Dependencies - list of packages that have to be build before building this package AliEn Packages Depending on it - the list of packages that can't be build before building this package

  11. Latest installation instructions for the user distribution can be found at http://alien2.cern.ch wget http://alien.cern.cern/alien-installer chmod +x alien-installer ./alien-installer [pcepalice10] /home/alienmaster > ./alien-installer ###################### Automated AliEn user installer ################ Installing in the default directory: /home/alienmaster/alien If you wish to install in a specific directory do: ./alien-installer -install-dir Platform set as: x86_64-unknown-linux-gnu Waiting 10 seconds. Press 'Ctrl+c' to exit the installation........Starting installation Downloading user distribution...................................Done Extracting the files........................................Done Relocating the user installation..............................Done Installation finished! AliEn: user distribution • User distribution contains the necessary libraries for compiling ROOT with AliEn Grid support. • The detailed instructions can be found on the same page as above.

  12. AliEn Tests are performed after each build • Test the installation procedure • Perform different functional tests • Save the log files for the failed tests • Save the log files of all AliEn services during the test • Publish relevant logs AliEn Tests

  13. General information Main test steps and their results Results per test types Release Test Web Page Logs: - AliEn services logs - failed tests logs Global log of the build process

  14. http://alirootbuild.cern.ch:8889 • Same build process as AliEn • Checks for new releases or changes in the current development releases • After each build performs quality and functional tests AliRoot: build specifics

  15. AliRoot: web interface http://alirootbuild.cern.ch:8889

  16. http://alirootbuild.cern.ch:8889 Adding a new AliRoot release • new releases can be added using a web interface • For each AliRoot release we need to specify the dependencies • AliEn version • Geant3 version • Root version Available version list extracted from source repositories Release administration

  17. AliRoot benchmarks are triggered after each build • The results are published on a web page • The logs are available from any remote location • History charts to compare with previous builds / releases AliRoot benchmarks

  18. After a complete build and in case of no errors during the tests, AliRoot packages can be registered into AliEn using a web interface from MonAlisa AliRoot: on the GRID

  19. Geant4 unlike its predecessor is written in C++ and has a completely new installation procedure • AliRoot BITS for Geant4 packages was adjusted in order to facilitate automatic compilation and installation procedures The Grid AliRoot: Geant4 • The final Geant4 package installed on the GRID (see left diagram) contains a number of dependent packages, libraries and data files. Geant4-v2010xxxx Geant4_vmc Data Files Geant4 CLHEP ROOT Build dependency Installation point

  20. Perl upgrade from 5.8.8 to 5.10.1 • Always use the latest packages • Enabling thread support for improving AliEn performance • Issues • Some packages are really old and they are not supported anymore • Rebuilding a distribution takes ~a couple of hours thus each iteration is slow • Problems are really difficult to spot • Packages that don't have a clear building and installation script • Because of the big number of packages and dependencies one change in a package can trigger a whole chain of problems • Some packages are not supported at all for some platforms, usually for Mac AliEn: current work [1]

  21. Adding a gLite test environment, without installing gLite with AliEn • Issues: installing gLite on all build servers independent of the build and testing process, certificates etc • Synchronizing the build number between different platforms • You can have AliEn 2-18.23 for 32bits and AliEn 2-18.30 for 64bits • Major cleaning of AliEn packages (somewhere in the future) AliEn: future plans

  22. Usual work: • Building and registering to AliEn different tags of AliRoot • Current work: • Implementing an automatic email alert system • If AliRoot fails, check where it failed and email the responsible persons about it • This applies only to the trunk, that is build every night • We are in test phase • Future plans : • Automatically register packages to AliEn, now it is done using an intermediate MonAlisa webpage AliRoot: present and future

More Related