1 / 23

SPI Report for the LHCC Comprehensive Review

SPI Report for the LHCC Comprehensive Review. Stefan Roiser for the SPI project. SPI services. Software process support Software testing, configuration & release Documentation support Code documentation, web services Collaboration tools Hypernews, Savannah Hardware infrastructure

twiley
Download Presentation

SPI Report for the LHCC Comprehensive Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. SPI Report for the LHCC Comprehensive Review Stefan Roiser for the SPI project

  2. SPI services • Software process support • Software testing, configuration & release • Documentation support • Code documentation, web services • Collaboration tools • Hypernews, Savannah • Hardware infrastructure • Build servers, special purpose machines LHCC Comprehensive Review - SPI

  3. Software

  4. Time to user <- Deployment <- Release <- Building / Testing <- -> Bug report / Feature request -> Implementation • The time from a bug report / feature request until it is used by the physicists in the experiments • Especially important to minimize this time when running with “live” data. LHCC Comprehensive Review - SPI

  5. Nightly Builds • Recommendation of the 2006 internal review • Motivation • Early integration into experiment software • Fast feedback to LCG/AA developers of changes • Usage • Building with own experiment nightlies on top • Developers testing their code on LCG/AA stack • Used by • LHCb • Atlas LHCC Comprehensive Review - SPI

  6. Nightly Builds (ctd.) 5 configurations Builds & Tests 13 possible platforms LCG/AA projects test evolution LHCC Comprehensive Review - SPI

  7. Software release Experiment software Experiment software eg. Athena, CMSSW, Gaudi eg. Athena, CMSSW, Gaudi LCG / AA software LCG / AA software External Software External Software Operating System Operating System Linux (slc) 32/64 bit, Mac OSX, Windows Linux (slc) 32/64 bit, Mac OSX, Windows LCG release • rubberstamps all package versions • provides several products • afs installation • web pages • src tar balls • bin tar balls • documentation COOL, CORAL, POOL, RELAX, ROOT, SEAL 56 packages, eg. Boost, Python, Qt, etc… LHCC Comprehensive Review - SPI

  8. Software processes historically grown nightly builds web sites docu python mysql cron / python tar balls xml xml / python install. mgr. cmt scram extern s/w project s/w LHCC Comprehensive Review - SPI

  9. Software processes new nightly builds web sites docu Python / LCGCMT tar balls install. mgr. extern s/w project s/w LHCC Comprehensive Review - SPI

  10. LCGCMT • Is a CMT based project • Contains the configuration information • How to build software packages from source • Using sh/bat scripts • Extra information • Eg. homepage, package maintainer, description, etc. • In addition for other CMT based projects • How to use/build against the packages • Runtime environment LHCC Comprehensive Review - SPI

  11. Example: package build package gsl use LCG_BuildPolicy v* LCG_Builders macro gsl_build_config_version "$(GSL_config_version)" set LCG_tarfilename "gsl-$(gsl_build_config_version).tar.gz" apply_pattern buildscripts_local #!/bin/sh cd ${LCG_builddir} tar xvfz ${LCG_tarfilename} cd gsl-${LCG_package_config_version} ./configure --prefix=${LCG_extdir}/GSL/${LCG_package_config_version} /${LCG_CMTCONFIG} #!/bin/sh cd ${LCG_builddir}/gsl-${LCG_package_config_version} make #!/bin/sh cd ${LCG_builddir}/gsl-${LCG_package_config_version} make install lxplus:~ >setenv CMTCONFIG slc4_amd64_gcc34 lxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_get lxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_config lxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_make lxplus:~/LCGCMT/LCG_Builders/gsl/cmt >cmt pkg_install LHCC Comprehensive Review - SPI

  12. Benefits of new s/w process <- Deployment <- Release <- Building / Testing <- -> Bug report / Feature request -> Implementation Current configuration LCG 53f has been built with these tools • Avoid duplication of information • Easier maintenance / automation • All parties use the same info (nightly builds, release, users rebuilding the software) • Speedup time to user LHCC Comprehensive Review - SPI

  13. Communication

  14. Hypernews • Bulletin board system used by LHC experiments • Moved to new IT provided hardware • 1 production machine, 1 hot standby • Helping with instance maintenance • Managing user DB, setup of for a • Atlas and CMS heavy users • Also used in Alice and Totem • LHCb is rethinking their usage LHCC Comprehensive Review - SPI

  15. Savannah • Savannah is the main bug reporting system in LCG/AA • Moved to new IT provided hardware • Used by many other parties (IT, Grid, etc.) • Reasonable system / user maintenance LHCC Comprehensive Review - SPI

  16. Hardware

  17. Hardware • LCG / AA build servers • 4 linux machines (managed by IT/FIO) • 2 mac osx machines (one ppc) • 2 windows machines • Special purpose machines • Web / database servers • Recycling old Hypernews machines (located in CC) • New machines for • Savannah, Hypernews LHCC Comprehensive Review - SPI

  18. Staff & Program of Work

  19. Staff • Change of project leader • Andreas Pfeiffer left to CMS • Stefan Roiser took over • 1 staff left, 1 fellow joined • Manuel Gallas left for Atlas • Piotr Kolet joined • 1 summer student • Luana Micallef • Currently 3.2 FTE LHCC Comprehensive Review - SPI

  20. Program of work • Completion of the integrated software configuration / testing / release • Web pages derived from LCGCMT • Integrate remaining release steps • Consistency checking • Production of distribution kits • Documentation generation • Completion of integration with nightly builds • Integration of installation manager LHCC Comprehensive Review - SPI

  21. Program of work (ctd.) • Nightly build system • Integrate Geant4 builds • Using IT infrastructure for • Setup of common LCG/AA web services • Setup of database services • Virtual windows machines for s/w build LHCC Comprehensive Review - SPI

  22. Program of work (ctd.) • Hypernews • Investigate usage of IT provided SSO • Find solution for longterm support LHCC Comprehensive Review - SPI

  23. Conclusion • Current focus on LCG/AA s/w stack • Optimize release process • Facilitate and unify processes for • Nightly builds • Release procedure • User rebuilds • Outsourcing of maintenance for other services, but still a lot of SPI manpower involved • Savannah • Hypernews • Hardware LHCC Comprehensive Review - SPI

More Related