1 / 29

Applications Area Status

Applications Area Status. LCG Mini-Review 16 th February 2009. Pere Mato/CERN. Outline. Applications Area Overview Status and Plans of the Projects Resources Summary. Applications Area Organization. MB. LHCC. Alice. Atlas. CMS. LHCb. Work plans Quarterly Reports. Reviews

HarrisCezar
Download Presentation

Applications Area Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applications Area Status LCG Mini-Review 16th February 2009 Pere Mato/CERN

  2. Outline • Applications Area Overview • Status and Plans of the Projects • Resources • Summary Applications Area Status

  3. Applications Area Organization MB LHCC Alice Atlas CMS LHCb Work plans QuarterlyReports Reviews Resources Chairs Architects Forum AA Manager Application Area Meeting Decisions LCG AA Projects ROOT PF SIMULATION SPI WP1 WP1 WP2 Subproject1 WP2 WP2 WP3 WP1 WP1 External Collaborations Geant4 EGEE ROOT Applications Area Status

  4. AA Projects • SPI – Software process infrastructure (S. Roiser) • Software and development services: external libraries, savannah, software distribution, support for builds, tests, QA, etc. • ROOT – Core Libraries and Services (R. Brun) • Foundation class libraries, I/O libraries, math libraries, framework services, dictionaries, scripting, GUI, graphics, data analysis, etc. • PF – Persistency Framework (A. Valassi ) • Storage manager, file catalogs, event collections, relational access layer, conditions database, etc. • SIMU - Simulation project (G. Cosmo) • Simulation framework, physics validation studies, MC event generators, participation in Geant4, Fluka. • PH White Paper R&D Projects • Exploit Multi-core architectures (V. Innocente) • Portable Analysis environment using Virtualization (P. Buncic) New New Applications Area Status

  5. Unchanged Execution Approach • Architects Forum (AF) meetings • Decision and action taking meetings • Experiments participate directly in the planning, management, and architectural and technical direction of AA activities • Every two weeks, public minutes after internal circulation • http://lcgapp.cern.ch/project/mgmt/af.html • Application Area meetings • Informal forum of exchange of information between the AA projects and experiments, etc. • Lower frequency than before • Workplans, quarterly reports and reviews • Discussed and agreed in the AF Applications Area Status

  6. LCG/AA Software ~62 packages • Continued with one/two major complete releases per year and a number of partial releases during the year to incorporate bug fixes/required new functionality • LCG_54 was first released January 2008 (+8 patches) • LCG_55 (de-sealed) June 2008 (+3 patches) • New web page with info configurations • http://lcgsoft.cern.ch • LCG_56 was released last week • New versions of all projects (ROOT, POOL,COOL,…) and many changes in ‘externals’ • To be used for this year running • Configuration and Release managementdone centrally by SPI project Applications Area Status

  7. AA Software Platforms • Introduced recently new compilers/platforms • Decisions always taken by the Architects Forum • SLC5 – 32/64 bit – gcc 3.4 / 4.3 • Windows Visual Studio 2008 (VC9) • All new AA software tested/released for many combinations • In total 20 combinations of operating system (SLC4, SLC5, OSX 10.5, Windows), compilers (gcc-3.4, gcc-4.3, VC7, VC9) and architectures (32, 64 bits) • What platforms and when to use them in production is left to the experiments • The infrastructure is in place to facilitate the platform management without too much effort Applications Area Status

  8. SPI - Collaboration Services • Hyper-News • Migration of the ATLAS instance to the new IT provided e-groups/Sharepoint service • Other experiments/projects have shown interest to move as well • Savannah • Stable and highly used service by several communities • LHC experiments, LCG-AA, CERN IT, Grid developers • Statistics in the last month • 101 new users, 5 new projects • 1865 new items (bugs, tasks, …), including 836 already closed • Overall • 4194 users, 297 projects • 63604 items, including 11711 still open Applications Area Status

  9. SPI - Nightly builds • Builds of the complete AA software stack are done every day for different ‘configurations’ and all ‘platforms’ • Tests are run systematically • Highly used service in the community • In use by LHC experiments to run validation and integration tests • Used by LCG/AA developers for integration tests • New developments • Multi-threaded builds on many core build servers • Considerable decrease of the total build time (factor 2) • Nightly system itself is also used and developed by LHCb & Geant4 Applications Area Status

  10. ROOT Highlights • The version of ROOT 5.22 was released on December 18th • This is the baseline version to be used by the LHC experiments • It includes a long list of new features and improvements • Re-structuring of the source repository reflecting better the structure of the project in terms of work packages and to easy the maintenance of the release notes and other documentation • first release of CINT (7) based on Reflex made available for testing to the experiments • Improvements in speed in many areas, size of dictionaries, etc. • Enhanced “schema evolution” ready for testing • ‘Consolidation’ is the most common word to describe the current project status Applications Area Status

  11. ROOT – Details (1) • Core (F. Rademakers) • support for SVN, installation, distribution • move from CVS to SVN -- a very successful operation • support for all Linux(es), MacOSX and Windows • restructuring the web site using the DRUPAL cms • I/O (R. Brun, Ph. Canal) • consolidation of the I/O -- support very complex C++ types  • automatic schema evolution enhancements • more functionality in Trees and selectors for data analysis • CINT/Reflex (A. Naumann, Ph. Canal) • many enhancements in Reflex • CINT version 7 based on Reflex • ACLIC: automatic interface with compiler and linker  Applications Area Status

  12. ROOT - Details (2) • MATH (L. Moneta) • Consolidation of MathCore, MathMore • Consolidation of linear Algebra classes -- fitting: better structure • RooStats in collaboration with Atlas and CMS (based on RooFit) • PROOF: (G. Ganis) • In production in Alice • Many developments in collaboration with the xrootd developers • ProofLite for multi-core systems • Better schedulers, packetizers -- Robustness • GUI (B. Bellenot) • Consolidation of the GUI classes, several new widgets • Event recorder class used for QA and tutorials • New browser with html support Applications Area Status

  13. ROOT – Details (3) • 2-D graphics (O. Couet) • Consolidation, always new options required • Improved output for pdf, gif, png, etc. formats • 3D graphics (M. Tadel - 50%) • Consolidation of the Open-GL classes • development of EVE (Event display infrastructure) used by Alice, CMS and other experiments • Geometry (A. Gheata (Alice)) • Consolidation, better visualisation • Used by the VMC, event displays Applications Area Status

  14. ROOT Applications Area Status

  15. Persistency Framework • Components Overview • CORAL relational database access (ATLAS, CMS, LHCb) • Most active PF component: CORAL server project • COOL conditions database (ATLAS, LHCb) • Many performance optimizations in 2008, now less active • POOL object streaming and catalogs (ATLAS, CMS, LHCb) • Largely in maintenance mode since summer 2008 (code changes mostly driven by new ROOT versions) • New PF project leader (A. Valassi replacing D. Duellmann) • Largely new (and smaller) team - expertise transfer, task reallocation Applications Area Status

  16. PF Achievements in 2008 • Maintenance tasks • Removal of SEAL from all of POOL, CORAL, COOL • Most SEAL functionalities were moved into CORAL • Main activity in Q1/Q2 (LCG_55 release, June 2008) • Many new platforms (gcc43, vc9, slc5) and externals • Main activity in Q4 (LCG_56 release, February 2009) • Review of POOL usage in the 3 experiments • Components used by only one experiment are maintained by them • Dropped obsolete packages, moved others to new CVS repository • New developments • CORAL server - slow progress in 2008, design reviewed in December • Performance improvements and tagging features in COOL • Improvements in POOL collections package Applications Area Status

  17. PF plans for 2009 • CORAL • CORAL server and CORAL server proxy • Most immediate use case: configuration of ATLAS online HLT • Secure authentication (certificates) for online and offline users • Reallocated some manpower from COOL to CORAL server • Improve coverage and configuration of CORAL test suite • COOL • Connectivity features (transaction control, session sharing) • Partitioning of relational schema • POOL • Review and improve DataSvc (only used by CMS) • All PF projects • Move code repositories from CVS to SVN Applications Area Status

  18. Simulation Project - Structure Simulation Project (G.Cosmo) Coordination Geant4 collaboration Geant4 (J.Apostolakis) LHC-oriented development and support of Geant4 toolkit Fluka (A.Ferrari) LHC-oriented development and support of Fluka FLUKA collaboration Garfield (R.Veenhof) Development, support and interfaces Experiments Validation Physics Validation & Simulation Framework (A.Ribon) Validation of the detector simulation MC engines Generator Services (A.Ribon) Event generators repository, generator validation, HepMC MC4LHC Applications Area Status

  19. Geant4: Releases, Highlights (1/2) • SFT members contribute in physics (validation and modeling), geometry and integration testing • Contributed to releases and patches • Releases 9.2 (Dec 2008), 9.2-beta (Jun 08), 9.1 (Dec 07) • Patches (3) were provided to release 9.1 during all of 2008 • As agreed with users (incl. LHC experiments) • Exceptional patches to 8.3 and 9.0, back-porting fixes from 9.1 patch 1 (Feb 2008) • Delta Review of Geant4 held 19th-20th Jan 2009 @ CERN • Panel included 3 physicists from LHC experiments • Major changes in integration testing • I. McLaren retired, and G. Folger took over • Undertook move to SPI nightly system for testing • Geometry: new GDML integrated module for reading and writing • EM processes: improved Bremsstrahlung • Relativistic corrections; new process for hadrons Applications Area Status

  20. Geant4: Highlights (2/2) Hadronic Physics Improvements • Hadronic physics is major focus area • Extended validation against further data • For energies around 1 GeV, from 5-15 GeV and at 150 GeV • Improved FTF Fritiof model (re-tuning) • For use in most challenging energy regime: 7-15 GeV • Created new physics lists (FTF_BIC, QGS_BIC) • Improved agreement in rapidity ‘region’ of target • From the re-interaction inside the nucleus of the slow particles created in high-energy part of interaction • Problem in matching of energy response remains • Reported by CMS, ATLAS. Reproduced in simple setups • Ongoing investigation Applications Area Status

  21. LCG Generator Services • GENSER • Structure stable and used by experiments • 24 generators installed (most with different versions) • Available for testing the first version of Pythia built with autotools • First version of a “bootstrap” script to help building GENSER automatically on a laptop • Started migration to SLC5 • 1 new testing package: MC-Tester • Started evaluation of Rivet and HepMC Analysis Tool for regression testing based on distributions • HepMC • New release 2.05.00 in progress, expected in March • MCDB • Under testing in CMS production Applications Area Status

  22. LCG Physics Validation • Pion longitudinal shower shapes are ok now;some minor improvements still needed for lateral shapes, and for protons (see LCG note: CERN-LCGAPP-2008-01) • Main focus now on energy response and resolution • Non-smooth transitions between hadronic models • QGSP_BERT is the default Physics List used in ATLAS and CMS. However, there is a growing interest for FTF_BIC and FTFP_BERT Physics Lists, after recent major improvements in the Fritiof model • smoother energy response than QGSP_BERT • but higher response than QGSP_BERT Applications Area Status

  23. CMS ECAL+HCAL response: E_vis / E_beam as a function of the beam energy, for pion and proton beams S. Banerjee, LCG Physics Validation meeting, 3-Dec-2008 Applications Area Status

  24. WP8 – Multi core R&D • Goals • Investigate current and future multi-core architectures • Measure and analyze performance of current LHC physics application software on multi-core architectures • Investigate solutions to parallelize current LHC physics software at application framework level and also investigate solutions to parallelize algorithms • Status • Benchmarked new Intel core i7 architecture with particular attention to hyper-threading • Evaluation of KSM Linux kernel memory module. Memory usage reduced by factor 2 in realistic programs (ATLAS, CMS) • On going work on multi-threaded Geant4, ProofLite, Parallel GAUDI with Python, etc. • On going work in parallelization of Minimization algorithms Applications Area Status

  25. WP9 – Virtualization R&D • Development of a Portable Analysis Environment using Virtualization Technology • Evaluation of the available virtualization technologies (virtual appliances) • Deployment of a read-only distributed file system with aggressive caching schema • Status • First version of CernVM released last October after a sequence of development releases • Compatibility with mostpopular hypervisors: • VMWare, Xen, KVM, Parallels, VirtualBox • 800 downloads Applications Area Status

  26. Manpower Table (excluding R&D) Applications Area Status

  27. Estimated Resource Needs Applications Area Status

  28. Staffing Remarks • Total reduction as (more or less) expected in 2009 • A couple FTE could be recovered later in the year (Fellow + PJAS) • Development of some unbalance between projects out of our control (LD->IC) • Simulation activities are very much affected • The current total 8.9 FTE should be compared to 14.8 in Nov 2007 • Several milestones in Physics Validation on hold • Geant4 testing also affected • PROOF development is not at risk anymore Applications Area Status

  29. Summary • Applications Area has continued to provide the software and services required by the experiments • The organization is mature and works reasonable well • Software is ready for this year data-taking • Improved release procedure of the complete software stack • Managed by SPI project with good collaboration from experiments to validate software before release • Software releases (fixes) can be done very quickly on demand • Reduction/Unbalance of manpower is affecting seriously some of the projects such as Simulation • The two R&D work packages have started to produce some results • Good collaboration between experiments Applications Area Status

More Related