CLEO3 software on Linux laptop

1. Valentine Kouznetsov Cornell University Outlines: Introduction to Linux New way to deploy Cleo3 software Introduction to RPM Cleo3 RPMs Demonstration Summary CLEO3 software on Linux laptop

2. Introduction to Linux • Linux was invented by Linus Torvalds (Finland, 1991) • It runs on: Intel/AMD/Cyrix, DEC alphas, Sun, SGI, IBM mainframe (@server), PowerPC, Macs, Amiga, ... • It can fit into: floppy, CDROM, ZIP, ROM, RAM, hard drive • It can run on variety of hardware including laptops (82+ different vendors www.linux-laptop.net) and embedded products like mobile phones, MP3 players, PDA, TiVO (http://www.linuxdevices.com) • Number of Linux distributions around the world: 345 (www.lwn.net), but if you need you can make your own one • Number of Linux users: 18 millions (www.linuxcounter.org) • Number of docs mentioned Linux: 56 millions (www.google.com) • It used everywhere, from science till motion picture production (Titanic, Shrek) • It used by anyone, from my son till German government • It is free and open source

3. New way to deploy Cleo3 software to Linux (automatic installation using RPM format) • We are moving towards Linux and need to find a way to deploy cleo3 software to our collaborators • We have multiple releases and multiple updates will be necessary • Installation/upgrade should be very simple, ideally “press a button” and drink your coffee while watching installation/upgrade process • RPM format is de facto standard to deploy software on Linux • It's easy to upgrade/verify/install software • User friendly, no specific knowledge is required • Guaranteed build (while packing RPM we verify that release was build and tested) • We pack third party packages (Tcl, readline, ROOT, CERNLIB) for you using cleo3 specific compiler flags (-fexceptions) • If you don't like RPM format it's easy to convert it to another formats using alien (http://kitinet.net/programs/alien): cpio and tar (Unix-like utilities), deb (Debian), tgz (Slackware) Requirements Solution

4. RPM introduction • RPM stands for Red Hat Package Management system • Adopted/used by almost all Linux distributions • Can be installed on ANY Linux/Solaris/OSF/AIX/SGI systems • In simple words RPM is a Linux install manager, rpm files are analogous of self extracting exe files on Windows and/or image files on Macs • There are two type of RPMs files: • Binary RPMs are pre-build ready to go packages, can be installed straightaway on your system. They keep a lot of information where/when package was built, package description, dependencies, etc. • Source RPMs contain source code (in tar.gz format) and spec file (instruction how to build package on your system) • Red Hat, SuSE, Mandrake etc distributions use RPM • www.rpmfind.net is largest RPM database (for all architectures and Linux distributions) • Currently almost 155K rpms are available on this site

5. Understanding RPM • RPM is powerful command line driven package management system capable of installing, uninstalling, verifying, querying and updating computer software packages. • It uses Berkley DB for querying and keeps snapshot of your system • It provides variety of information about package • Package-wide information: time stamp package was built, description of package's contents, total size of all files installed by the package, digital “signature”, package dependencies • Per-file information: name of every file and where it is to be installed, each file's permissions, each file's ownership, MD5 checksum of each file, the file's contents • RPM file naming convention name-version-release.architecture.rpm • name is a name describing the packaged software • version is the version of packaged software • release is the number of times this version of software has been packaged • architecture is shorthand name describing the type of computer hardware the packaged software is meant to run on. • Example: am-1.0.0-1.i386.rpm, am-1.0.0-1.src.rpm, am-1.0.0-1.alpha.rpm

6. Verifying RPM • RPM is powerful tool to examine integrity of your installed software • There are nine attributes RPM is capable to catch: • File size (S) • File ownership (U) • Group ownership (G) • File mode, “permission” (M) • MD5 checksum is a 128-bit number derived from file contents (5) • Major/Minor number is device character and block file information used by different drivers on the systems (D) • Symbolic link is the text string containing the name of the linked-to file (L) • Modification time (T) • Dependencies and missing files • SM5DLUGT c filename [vk@localhost vk]$ rpm -qf /nfs/cleo3/Offline/rel/current-1.0.0/bin/Linux/g++/suez cleo3-suez-1.0.0-1 [vk@localhost vk]$ rpm -V cleo3-suez-1.0.0-1 Unsatisfied dependencies for cleo3-suez-1.0.0-1: /usr/local/bin/bash, /usr/local/bin/tclsh, libtcl8.2.so S.5....T /nfs/cleo3/Offline/rel/current-1.0.0/bin/Linux/g++/suez .......T /nfs/cleo3/Offline/rel/current-1.0.0/build/lib/.release

7. Using RPMs • There are command line and GUI RPM interfaces • Install/Upgrade and erase • rpm -ivh or -Uvh • rpm -e • Query • rpm -qa | grep <your_RPM> • rpm -qf /bin/ls • rpm -ql cleo3-mcinfo-1.0.0 • Building • rpm -bb (for binary) rpm -ba (for source/binary rpms) • Verify • rpm -Vf /bin/ls

8. Cleo3 requirements • RedHat 9.0 or any other Linux distribution with following packages: • rpm version >= 4.2, gcc version >= 3.2.2, glibc version >= 2.3, am-utils • There is no requirements where to install cleo3 software • With root access you need 3Gb on /nfs/cleo3 and 2-4 Gb on / • Without root access you need 3 (plus 2-4) Gb on your disk for cleo3 software • We provide only source RPMs • Due to explicit path requirements you need to rebuild your shared objects and executables with respect to your local installation path • It's easy to transfer few Mb rather then several Gb • Installation script create local copy of RPM DB therefore cleo3 RPMs are separated from default RedHat RPMs • Cleo3 RPMs naming convention: • cleo3-<package_name>-<releaseversion> • Cleo3 dependencies • Cleo3-mcgen requires: cleo3-Gmake, cleo3-suez, cleo3-others, cleo3-mcinfo

9. Cleo3 release(s) & RPM(s) • We add rules to Cleo3 release structure: • Release version: 1.0.0 (any time stamp or number) • Shared objects (*.so) inherit release version: libProcessor.so.1.0.0 • Executables are linked to load shared objects of particular version: ldd suez.exe /nfs/cleo3/Offline/rel/current-1.0.0/lib/Linux/g++/libProcessor.so.1.0.0 /nfs/cleo3/Offline/rel/current-1.0.0/lib/Linux/g++/libDataDelivery.so.1.0.0 libgcc_s.so.1 => /lib/libgcc_s.so.1 • RPM version corresponds to release version, e.g. cleo3-suez-1.0.0-1.i386.rpm means cleo3-suez package was packed from release 1.0.0, 1st build on Linux platform (i386) • You may have multiple releases on disk and different combinations may coexists, e.g. two off-line releases may use one version of other_sources • Cleo3 packages grouped into logic structure packages (currently 19 RPMs)

10. CLEO3 RPM organization • There are third party packages required by CLEO software • CLHEP, Tcl/Tk, readline, Root, CERNLIB, mico • Offline scripts to setup CLEO environment • common (Online) software • suez • mcinfo – set of common packages for high level ones • mcgen – set of packages to generate MC 4 vectors • cleog – set of packages to generate geant'ed MC • pass2(data,cleo2) to perform pass2 reconstruction Cleo3 RPM dependency graph

11. Cleo3 RPM statistics & achievements • I was able to build/install cleo3 RPMs on this laptop (Red Hat 9) • 256 Mb of RAM, 730 MHz • I tested two desktops running Red Hat 8 at Cornell and Minnesota (patches required) • Build time varying from few minutes till 3 hours per package • Total time spent: < 12 hours • I was able to run • MC generation (4 vectors) BBbarGenerationProd • Cleog step (50 events) • Pass2 generated MC and real data, run 128373 (250 events) • RPMs installed on Minnesota farm: • 14K MC events (cleog+pass2) Source RPMs 16 Mb Binary RPMs 350 Mb Release area 2.8 Gb

12. Need to be done Access to Cleo3 constants DB Through cache servers Installation tune ups Eliminate cleo2 dependencies reduce size of RPMs static vs shared libraries Cleo3 RPMs benefits & unresolved issues Benefits • Eliminate user errors during installation • Automatic installation of ALL necessary software compiled with appropriate Cleo3 flags • Modularity: you can install only necessary RPMs depending on your goal (generate MC, run suez/pass2, perform analysis) • Software management • If something wrong on your system you can verify integrity of your software and install/erase/update it • Keep track of multiple releases • Patching version of cleo3 releases

13. Installation procedure • Automatic installation script is provided • cleo3_install_rpm.sh (can be found in C3MakeRpms package) • Guide you through installation process • It creates local version of RPM db in your desired location • It is defined wrt to your path: <your_path>/rpm • Cleo3 RPMs reside in separate RPM db and cannot harm your system • No root access to your system is required But you may install Cleo3 RPMs in default RPM db using root account • Due to dependency requirements you will not able to build high-level package until you'll install low-level packages, e.g. If cleo3-suez is not installed you cannot build cleo3-trkobj AND installation script takes care of that • You may build/install only those RPM which are suitable for your goal (to generate MC or analysis) • To erase Cleo3 package: • rpm --dbpath <your_path>/rpm -e cleo3-XXX-1.0.0-1 • Try, test it and send me your feedback vek3@cornell.edu

14. Manual installation procedure • Automatic installation script is provided • cleo3_install_rpm.sh • Manual installation requires a few steps: • Create ~/.rpmmacros with %_topdir <your_path> %_tmpdir <your_path>/tmp • Create directory structure to perform RPM build: <your_path>/{BUILD,RPMS/i386, SRPMS,SOURCES,SPECS} • cp -r /var/lib/rpm <your_RPMDB_path> • Setup your environment C3_, C_, C3_INSTALL_DIR • Copy source RPM somewhere on your system and invoke the following command rpmbuild –dbpath <your_RPMDB_path> –rebuild cleo3-XXX-1.0.0-1.src.rpm • Install your binary RPM • rpm -ivh –dbpath <your_RPMDB_path> --prefix <your_path> <your_path>/RPMS/i386/cleo3-XXX-1.0.0-1.i386.rpm • If something going wrong look at automatic script for details

15. Demonstration

16. Summary • First version of CLEO3 RPMs is available for public usage • /home/vk/dafj/Cleo3_rpm • cleo3_install_rpm.sh • We successfully build 19 RPMs, install them on test laptop and desktops at Cornell and Minnesota • Using this RPMs we're able to run: • MC generation (4-vector) • Cleog and pass2 MC (50 events) • Pass2 real data, run 128379 (250 events) • Minnesota farm produced 14K events based on RPM cleo3. • You can install appropriate set of RPMs on your system depending on your goal • Installation procedure is very simple • No other software required (we pre-build third party packages for you, including necessary options, e.g. -fexceptions for C packages to allow suez catch exceptions).

17. Conclusion • Linux is coming and we're prepared to deploy Cleo3 software to your Linux box • Everything you saw today was done under Linux and using Linux • Linux is not out of the box product instead it is a lego constructor, you can make it nice, stylish, cool, hot, ugly,... • Don't be afraid to use it, but be prepared to fix something if something is going wrong You can run/develop your cleo3 software on laptop laying down on Florida beach or watching safari It can be real fun !!!