Software Processes in BaBar

1. Software Processes in BaBar Gabriele Cosmo CERN - IT/ASD Gabriele.Cosmo@cern.ch 3rd LHC Computing Workshop Marseille, September 28th- October 1st,1999

2. Outline • The BaBar environment • Training and courses • Tools for Software development • Documentation and HyperNews • The Release mechanism • The User Environment • Conclusions

3. The BaBar environment • More than 70 collaborating institutes • distributed environment and computing resources • Currently supported architectures/OS: UNIX • SUN, OSF, (in a near future also Linux) • Programming Languages: • C++, C, Fortran • wide spectrum of expertise • need for proper training to OO Programming

4. Training and courses • OO Methodology: Object Mentor Course • Principles of UML and OOD • Advanced OOAD and Design Patterns • See also: www.oma.com/Brochure/training.html • Course on C++ programming language • Courses on specific BaBar software applications • Organized regularly according to BaBar collaboration meetings schedule

5. Tools for Software Development • CASE tools: Rational-Rose • identified a team of experts for helping in designing • Software QA tools: Code Wizard • Run-time checkers • Insure++ , Great Circle, Purify • Special builds regularly performed over “stable” or “production” releases • results automatically posted to the package coordinator • summary published on Web

6. WWW and HyperNews • Distributed environment • WWW plays a fundamental rule ! • Software standards, requirements & polices • Meetings, minutes, slides, tutorials, ... • Calendars, release schedule, computing resources info • HyperNews: discussion forum organized by topic • uses BFMAIL • users are free to subscribe or unsubscribe • possibility to add new topics • the Release mechanism is integrated • a special topic: authoritative software recipes

7. Documentation • FAQ: to focus on the most common problems • Workbook: collects docs for users & developers • LIGHT: code browsing & navigation • Code documentation • LaTeX, FrameMaker, HTML • Cross Reference • KDOC, Javadoc

8. Production QA • Dedicated QA team & QA tools (sim & reco) • at every new release: reference data from old release are compared and the new reference set is updated • A set of references is needed for each different run type which the QA Production Tools use in their comparison tests • histograms & data • automated mechanism in production • main results published on web

9. SRT - Software Release Tools • Originally developed in BaBar • by Bob Jacobsen, first release in November 1994 • Based upon CVS: the Version Control System • the actual code repository • Uses GNU Make • It’s a Package ! • set of GNUmake scripts (Tools)

10. SRT: What is all about ? • Set of tools (GNU make scripts) useful to: • define the release environment • build software releases on different architectures and compilers in a homogeneous manner • define common rules used in the GNUmakefiles • DB setup, schemas, libraries, binaries, docs, tests • provide utilities to control and make easier the release process in a distributed environment • importrel, importver, updrel, rmrel, auditrel, statusrel, listtag, ...

11. Packages • Package: • a logical collection of classes/functions providing a set of services and performing a well defined task (Booch Category) • Maintained as a separate CVS module • Contains: code, docs, a GNUmakefile, scripts… • A package coordinator (or more) is responsible for: • the package development and testing • tagging and announcing the package for the release

12. Versioning and Tagging • CVS is used for versioning • Fixed tag format, i.e. V01-25-08[-02] • Tags are also used as names of subdirectories • A tag can survive to more than one release until a new tag is announced • Request for creation of a new package must take place some time before the scheduled release, specifying: tag, coordinator, README file

13. The BaBar Software Directory tree • Access to CVS modules can be controlled via AFS ACLs if required. • Package coordinators are notified automatically when new code is checked in.

14. A Release in BaBar • A snapshot of a consistent set of package versions (online, offline, utilities, scripts, makefiles, …) • a general rule: never use the CVS HEAD ! • It includes libraries and binaries for each BaBar supported platform • more than 300 libraries and 80 executables • Built every two weeks. Indexed by a release number • i.e. - release 8.2.7 • Test Release - a successful build • Current or Production Release - a stable tested release • Newest Release - the last available release

15. Release: the Working Model - 1 • Package coordinators release their package by testing, committing, tagging and announcing their code for a release and/or nightly build • Automated system based on Web, HyperNews and BFMAIL • Nightly builds start automatically each night on all supported platforms, collecting all announced tags. • problems are automatically posted to the PC the day after • Every two weeks, at the scheduled day & time, all tags are collected and the release process starts • Problems are automatically posted to the PC the day after • A summary of the build is posted together with notes by the Release Coordinator

16. Release: the Working Model - 2 • Automated testing • based on specific tests provided by the package coordinators • can be run automatically at release stage • output compared to supplied reference • Error from all release stages are automatically posted to package coordinators • problems not fixed are “gently” reminded every 7 days by the BaBar Problem Tracking System • Outside release builds, bug reporting and tracking is controlled through RemedyARWeb based on ORACLE

17. The User Environment - 1 • SRT requires few environment variables and entries in $PATH and $MANPATH variables • Automatically set via centralized HEPiX scripts • Test Release - the user copy of the original release structure for a specific version • newrel, addpkg • bin, lib, tmp, shlib, shtmp, doc, test, GNUmakefile

18. The User Environment -2 • Only packages added in the Test Release are built • The final executable is linked against libraries of Test Release and the correspondent SRT release • The user can add any version of a package • example: addpkg L3Trigger V00-03-11 • SRT’s rules can be extended for a given package • custom rules/targets in the package GNUmakefile • keep the SRT’s standard GNUmakefile body

19. Tools: Workdir • It’s a (simple) package • Includes a “special” GNUmakefile integrated in the SRT environment • Allows to the user to run applications from every area (even outside the user Test Release) • Ability to choose in a flexible way the release (or Test Release) to which refer to • Creates automatically links to major scripts

20. Tools: PackageList • It’s a package ! • Contains information pertaining to every package which is part of the BaBar software release system • example: PACKAGELIST += L1Sim ifneq ($(LINK_L1Sim),) override LOADLIBES += -lL1Sim -include L1Sim/link_L1Sim.mk endif • Requires a file link_$PACKAGE.mk in each package specifying its dependencies over other packages • creation can be automated • Objective: define a standard link order • Relies on good consciousness of the developer

21. The BaBar Application Framework - 1 • It’s a package ! • developed by D.Quarrie • Implements a Modular approach • a module is a fragment of executable code which has well-defined interfaces and perform a well-defined service • modules are totally independent each other, have their own configuration and data-taking run specific information • each module provides an interface to the Framework which includes a unique name and functions to be called at the beginning/end of the job, data-taking run, event modifier

22. The BaBar Application Framework - 2 • Modules are associated into Sequences • a sequence of modules is a list of one or more modules executed in the order they appear • the order of execution and the messaging of input run-time parameters for each module are controlled by Tcl scripts • the event is the central data-structure and is designed to be an “object dictionary” • other concepts: path, output stream, processor • Automatic regression-testing of a single framework package integrated in the system

23. Conclusions -1 • OO Methodology (UML) • Training and tutorials for developers • CASE tools • for analysis and design of software units • QA tools • integrated in the software environment • available for daily software development • Use of WWW and BFMAIL • documentation & communication (user/developer) • automated release mechanism

24. Conclusions - 2 • SRT+CVS+HyperNews • a working combination of products for Software Processes in a distributed environment of a modern HEP experiment • SRT: originally developed in BaBar, now adopted and constantly customized by more and more experiments world-wide: FNAL, ATLAS, CMS, CDF, D0, BTeV • requires an effort and a certain degree of responsibility by the user (Package Coordinator) to achieve the best use • offers a flexible and efficient user environment • can be easily extended with new tools

25. Sometimes, no matter how hard you work, you can’t meet management’s objectives