1 / 42

BOSS Development Environment BOSS-- B ESIII O ffline S oftware S ystem

BOSS Development Environment BOSS-- B ESIII O ffline S oftware S ystem. Ma Qiumei 2006.03.01. Outline. Simple introduction of BOSS How to use CMT in BOSS Simple introduction of CVS Common questions and solutions.

javier
Download Presentation

BOSS Development Environment BOSS-- B ESIII O ffline S oftware S ystem

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BOSS Development EnvironmentBOSS--BESIII Offline Software System Ma Qiumei 2006.03.01

  2. Outline • Simple introduction of BOSS • How to use CMT in BOSS • Simple introduction of CVS • Common questions and solutions

  3. History of BesIII Offline Software System • Besf/v1 : The original version completely based on belle • Besf/v2 : The version improving with other frameworks like babar, gaudi, etc. • Offline/3.0.0 : The version with the new building tools—CMT • Boss/1.0.0 : The testing version based on Gaudi • Boss/2.0.0 : The modified version based on Gaudi • Boss/2.1.0、2.2.0、2.3.0、3.0.0、4.0.0、4.1.0、4.2.0 5.0.0、5.1.0、6.0.0

  4. Boss development environment(Now) • Machine: lxplus.ihep.ac.cn • OS : Linux-7.3 • Compiler : gcc-3.2 • Framework :Based on GAUDI-v12r0 • External Libs: CERNLIB、ROOT、CLHEP、Geant4、and etc. • Developing language : C++，some FORTRAN • Database : PostgreSql,MySql • Version management tool : CVS • Browsing files in CVS repository :ViewCVS • Configuration management tool : CMT

  5. Boss development environment(future) • Machine: lxplus.ihep.ac.cn • OS : Scientific Linux Cern (SLC) • Compiler : gcc-3.2.3 • Framework :Based on GAUDI_v16r4 • External lib: lcg/external; geant4 and etc. • Developing language : C++，some FORTRAN • Database : PostgreSql,MySql • Version management tool : CVS • Browsing files in CVS repository :ViewCVS • Configuration management tool : CMT

  6. BOSS Structure Analysis -- Analysis software and some analysis tools BesExamples -- Examples for using some packages BesPolicy -- define some standard patterns of Bes BesCxxPolicy -- define some standard patterns only for c++ compilier BesFortranPolicy -- define some standard patterns only for fortran compilier BesRelease -- for all the packages release BesROOT -- convert data from ASCII to DST Calibration -- for calibration framework and common tools Control -- Common tools and framework of boss Database -- the geometry data IO

  7. DetectorDescription -- Detector Description packages Emc -- for emc Event -- different event model EventDisplay -- the interface for event display EventFilter -- External -- interfaces of external libs Generator -- for generators MagneticField -- Mdc -- for mdc

  8. Muc -- for MuonCounter PhyTool -- some tools for physics analysis Reconstruction -- all reconsrtuction algorithms RootPolicy -- Simulation -- simulation TestRelease -- for some test release Tof -- tof Utilities -- some common tools • More details please see the website: http://koala.ihep.ac.cn/cgi-bin/viewcvs.cgi/BossCvs

  9. How to use CMT in BOSS • Simple introduction of CMT • Setting up your CMT environment • Working on a package • Working with boss

  10. What is CMT（ Configuration management tool） • A set of tools and conventions • structures software development or production • concepts of areas, packages, versions, constituents • organises software into packages • describes package properties • describes package constituents • operates the software production (management, build, import/export, etc...) • by transparently configuring and driving the various conventional tools (CVS, make, MSDev, Web, tar, compilers, linkers, archivers, etc...)

  11. The goals • Organise software development from a single person up to teams in large projects (scalability) • Organise relationships between entire software bases or between simple packages (scalability) • Manage site, platform or product specific properties • Automate complex production tools for non software experts (simplicity) • Factor out know-how from project to project, from package to package or from team to team (process improvement)

  12. Who are using CMT • The physics research experiments using CMT as the following: a.. Virgo b.. LHCb c.. GLAST d.. Atlas e.. Nemo f.. Opera g.. Planck h.. Auger i.. GANIL j.. GAM

  13. Types of Package Packages may have particular semantics • Normal packages provide software or documents • Container packages manage other packages • Policy packages hold bes specific conventions, policies, patterns • Interfaceconvert conventions and policies of external packages or libs

  14. Structure of a package • /src source files (*.cxx) • /<package-name> header files (*.h) • /cmt requirements file and some scripts • /share jobOptions and other running scripts • /test some app’s source files • /i386_linux24Automatically created at build time (libs. Applications and etc. under this directory) • /ChangeLog a log file

  15. BesGeoMdc BesGeoMdc-00-* BesRelease BesRelease-00-00-01 use BesPolicy BesPolicy-01-* MDCGeomSvc MDCGeomSvc-00-00-01 use External/CLHEP CLHEP-00-01-* use use Release area use CLHEP use use use MDCGeomSvc MDCGeomSvc-01-00-01 External area User area Global software organization use

  16. 几个概念 • Release（发布；软件版本）：某个release对应于所有软件包的某一个固定版本，它是稳定的，一旦发布基本上就不再更改。 • nightly_build：不需要发布一个新的release，但是用户希望使用某些软件包的新改变，就建立了这么一个release：nightly_build。这个release是不稳定的，经常更新。 • TestRelease：运行boss的接口程序，可以选择需要运行的程序。

  17. Setting up your CMT environment • 首先，连接CMT • $mkdir cmthome • $cp /afs/ihep.ac.cn/bes3/offline/template/cmthome/setupCMT.csh ~/cmthome 注：如果你使用的是bash，则copy setupCMT.sh到你的 cmthome目录下。 • 然后，修改login requirements，设置符合自己要求的配置 • $cp /afs/ihep.ac.cn/bes3/offline/template/cmthome/requirements ~/cmthome • 编辑requirements文件，使其符合你的要求 $vi requirements 注：需要修改的可能有两个地方： set BES_RELEASE 5.0.0 macro WorkArea "/ihepbatch/bes/maqm/workarea"

  18. 最后，设置环境完成 • source setupCMT.csh • cmt config • source setup.csh 注：最好把以下语句放到.tcshrc文件中： • source /ihepbatch/bes/maqm/cmthome/setupCMT.csh • source /ihepbatch/bes/maqm/cmthome/setupCVS.csh • source /ihepbatch/bes/maqm/cmthome/setup.csh • source /ihepbatch/bes/maqm/workarea/TestRelease/TestRelease-00-00-14/cmt/setup.csh

  19. Working on a package • Basic scenarios • Creating a package • Checking out a package alone • Modifying the package • More: working on several packages • Apply same set of commands via cmt broadcast

  20. Creating a new package 1、Choosing a location • i.e. Which container? … e.g. Database 2、Choosing a name • Must be a unique package name in boss (should not either just differentiate by case of letters) 3、Choosing a version 4、The command cmt create BesGeoDB BesGeoDB-00-00-01

  21. >cmt create A A-00-00-01 >cd A/A-00-00-01/cmt >vi requirements >vi A.cxx >gmake >source setup.csh >cd ../i386-linux24 >./A.exe 1) Create a new package and move to it 2) Describe the package 3) Create the package items and build them 4) set the environment and run package A use BesPolicy BesPolicy-01-* application A A.cxx Examples… • The simplest one • One new application with (some) source file(s)

  22. The requirements file The requirements file centralizes all configuration parameters Make macros and environment variables and their possible values on various platforms, sites, environments Author(s), manager(s) Customization for new languages, or document generators • Constituents • Libraries • Applications • generated documents The requirements file • Structural information • specialized directory structure • used packages • links to external packages Definition of conventional behavioural patterns Sufficient for transparently generating all required makefiles and setup scripts, or for launching test procedures. They are the sources of all queries.

  23. Example of requirements package BesGeoDB author maqiumei maqm@ihep.ac.cn version BesGeoDB-00-00-01 branches cmt src BesGeoDB use BesPolicy BesPolicy-* use PGSQL PGSQL-* External library BesGeoDB *.cxx apply_pattern package_linkopts

  24. Checking out a package • Getting a package out from CVS to modify it • Go to your private development area • Checkout the package (installing the proper CMT directory structure) cmt co TestRelease • Get the HEAD revision, and make use of the most recent tag cmt co –r TestRelease-00-00-15 TestRelease • Get the required revision, and make use of that tag • The directory version is created with the appropriate CVS tag applied to the package

  25. Modifying and building a package • The basic development cycle • Modify > vi *** • Build > source setup.csh > gmake • Test (if available)

  26. Working on several packages • Checking out a complete container cmt co Database cmt co –requirements Database/*/cmt/requirements • Checking out all packages cmt co BesRelease cmt co –requirements BesRelease/*/cmt/requirements • Applying a command to all checked out packages cmt broadcast <shell command> • Cmt broadcast cmt config • Cmt broadcast gmake

  27. Working with boss • The principle of the TestRelease package • This package is the basic interface to run a job • Running boss only requires to “select” a set of already built components and to locally install a set of runtime files Checking out TestRelease • Customizing it : selecting components • Before running : install runtime files • This generally creates symlinks towards runtime files (shared libraries, jobOptions, data files, …) • Running boss

  28. Running boss #running a simple example -- ExHelloWorld • cd <yourworkarea> • cmt co TestRelease • cd Testrelease/*/cmt • vi requirements • Add “use ExHelloWorld ExHelloWorld-* BesExamples” to requirements • cmt broadcast cmt config • cmt broadcast gmake • source setup.csh • cd ../run • boss.exe HelloWorldOptions.txt • boss.exe jobOptions_rec.txt

  29. Simple introduction of jobOptions ////#################################### // Boss job options file for MdcRec //====================================== ApplicationMgr.DLLs+= {"MdcTrkRecon"}; ApplicationMgr.TopAlg += { "MdcTrkRecon" }; ApplicationMgr.ExtSvc += { "MdcGeomSvc" }; ApplicationMgr.DLLs += { "RootHistCnv" }; //-------------------------------------------------------------- // Set output level threshold (2=DEBUG, 3=INFO, 4=WARNING, 5=ERROR, 6=FATAL ) // MessageSvc.OutputLevel = 2;

  30. // Number of events to be processed (default is 10) ApplicationMgr.EvtMax = 200; EventSelector.InputFiles ="AsciiDmp_out.dat"; ApplicationMgr.HistogramPersistency = "ROOT"; NTupleSvc.Output = { "FILE101 DATAFILE='Mdc.root' OPT='NEW' TYP='ROOT'“}; //=============================== // End of job options file // ===============================

  31. Running the reconstruction algorithms #include "Ascii.txt“ #include "PartPropSvc.txt" //zhangy #include "$MDCTRKRECONROOT/share/jobOptions_MdcPatRec.txt“ //zangsl //#include "$TRKRECOROOT/share/jobOptions_TrkReco.txt" #include "$MDCDEDXALGROOT/share/job_dedx_all.txt" #include "$EMCRECROOT/share/EmcRecOptions.txt" #include "$TRKEXTALGROOT/share/TrkExtAlgOption.txt" #include "$TOFRECROOT/share/jobOptions_TofRec.txt" #include "$MUCRECALGROOT/share/jobOptions_MucRecTrkExt.txt" #include "$ROOTIOROOT/share/jobOptions_Save2Root.txt"

  32. // Set output level threshold (2=DEBUG, 3=INFO, 4=WARNING, 5=ERROR, 6=FATAL ) MessageSvc.OutputLevel = 5; // Number of events to be processed (default is 10) ApplicationMgr.EvtMax = 200; EventSelector.InputFiles="AsciiDmp_out.dat"; ApplicationMgr.HistogramPersistency = "ROOT"; RootCnvSvc.digiRootOutputFile ="digi.root"; NTupleSvc.Output = { "FILE101 DATAFILE='Mdc.root' OPT='NEW' TYP='ROOT'", "FILE104 DATAFILE='kalfit.root' OPT='NEW' TYP='ROOT'", "FILE103 DATAFILE='DeDx.root' OPT='NEW' TYP='ROOT'", "FILE201 DATAFILE='Tof.root' OPT='NEW' TYP='ROOT'", "FILE301 DATAFILE='Emc.root' OPT='NEW' TYP='ROOT'", "FILE401 DATAFILE='Muc.root' OPT='NEW' TYP='ROOT'" };

  33. CVS是什么? • 1989年4月，Brian Berliner设计了CVS并编写了代码，之后Jeff Polk帮助Brian设计了CVS模块发行分支。 • CVS(Concurrent Versions System )版本控制系统是一种GNU软件包，主要用于在多人开发环境下的源码的维护。 它可以记录源文件开发的历史。

  34. CVS and CMT • The BESIII Offline Software sources are stored in CVS and can be accessed using the CVS command. • The BESIII Offline Software libraries have been built using the CMT. Therefore, using the CMT tool is the recommended way to modify existing packages or re-build the examples included in the release.

  35. 如何使用CVS？ • 首先，你需要申请一个CVS账号 • 然后，需要设置变量： export CVSROOT=':pserver:maqm@koala.ihep.ac.cn:/bes/bes' 或 setenv CVSROOT ‘:pserver:maqm@koala.ihep.ac.cn:/bes/bes‘ • 最后，你需要知道一些常用的CVS命令（见下页）

  36. 常用的CVS命令(1) • Checking in to CVS cvs import –m “**” test Bes test-00-00-01 • 查看源程序的变化 cvs –n update • 添加一个源文件 cvs add 源文件名（路径要正确） • 删除一个源文件 cvs remove 源文件名（路径要正确）

  37. 常用的CVS命令(2) • 提交所作的修改 cvs commit –m “**” • 给一个包做tag cvs tag 版本号 • 查看当前工作的源文件与cvs库里源文件的区别 cvs diff

  38. Viewcvs简介 • ViewCVS是一个使用Python书写的查看CVS代码库中的 所有数据 的软件。它已经被很多的配置管理系统所使用， 也是现在开放源代码产品中与cvsweb一样可以选择的CVS辅助工具。 • ViewCVS架设了一个友好的WEB界面，通过WEB界面可以非常直观的浏览代码,查看图形化的修改记录。 http://koala.ihep.ac.cn/cgi-bin/viewcvs.cgi/BossCvs/

  39. 常见问题及解决方法 • 编译的时候： • $ cmt config > Warning : package CERNLIB CERNLIB External not found (requested by tester) • $ gmake >ihepbatch/bes/maqm/workarea/Calibration/CalibData/CalibData-00-00-04/CalibData/CalibTime.h: 14:34: facilities/Timestamp.h: No such file or directory >/usr/bin/ld: cannot find -lxerces-c

  40. 运行的时候： • boss.exe jobOptions_sim.txt libGaudiSvc.so: cannot open shared object file: No such file or directory • boss.exe jobOptions_sim.txt undefined symbol: ****

  41. Documents • http://www.cmtsite.org • http://atlas.web.cern.ch/Atlas/Groups/software/oo/tools/cmt • http://boss.ihep.ac.cn • http://koala.ihep.ac.cn/cgi-bin/viewcvs.cgi/BossCvs/

  42. Thank you!

More Related