Applications Area Plans for Phase 2

1. Applications Area Plans for Phase 2 AA Meeting/CERN 5 October 2005 Pere Mato/CERN

2. Outline • The planning exercise • Update on SEAL and ROOT merge • Projects, Milestones and Resources LCG Applications Area Plans

3. AA Program of Work Phase 2 • Last week the AA Planning Document was approved by the PEB • http://lcgapp.cern.ch/project/docs/AA_Program_of_Work_Phase2_final.pdf • This ends the planning exercise that started in May/June • The plan is consistent with the change proposals presented at the AA internal review in April • Defined about 80 level-2 milestones to follow progress of the projects • Accounted resources being invested in each project • Most problematic area in the plan has been to reach an agreement on the SEAL and ROOT project merge • Compromise found and described later in the talk LCG Applications Area Plans

4. Scope and Requirements • The overall scope for AA is the same as for LCG Phase I • establishment of the basic environment for software development, documentation, distribution and support • development of general-purpose scientific libraries, C++ foundation libraries, and other standard libraries • development of tools for storing, managing and accessing data handled by physics applications • adaptation and validation of common frameworks and toolkits provided by projects of broader scope than LHC, such as Pythia, Geant4 and Fluka. • The main requirements are also basically the same as for Phase I LCG Applications Area Plans

5. Principal Architecture Requirements • Long lifetime: support technology evolution • C++ today; support language evolution • Seamless distributed operation and usability off-network • Component modularity, public interfaces • Interchangeability of implementations • Integration into coherent framework and experiment software • Design for end-user’s convenience more than the developer’s • Re-use existing implementations • Software quality at least as good as any LHC experiment • Meet performance, quality requirements of trigger/DAQ software • Platforms: Linux/gcc, Windows/vc++,MacOSX/gcc, LCG Applications Area Plans

6. Under Revision AA Organization LCG Applications Area Plans

7. AA Projects • SPI – Software process infrastructure (A. Aimar -> A. Pfeiffer) • Software and development services: external libraries, savannah, software distribution, support for build, test, QA, etc. • ROOT – Core Libraries and Services (R. Brun) • Foundation class libraries, math libraries, framework services, dictionaries, scripting, GUI, graphics, etc. • POOL – Persistency Framework (D. Duellmann) • Storage manager, file catalogs, event collections, relational access layer, conditions database, etc. • SIMU - Simulation project (G. Cosmo) • Simulation framework, physics validation studies, MC event generators, participation in Geant4, Fluka. LCG Applications Area Plans

8. SEAL and ROOT Merge • The main change described in the AA plan document is the merge of the SEAL and ROOT projects. • The change was proposed, reviewed in depth and subsequently endorsed in the AA internal review that took place at the beginning of April. • The main motivations for this merger have been the following: • to optimize use of resources by avoiding duplicate development • to provide a coherent set of products developed and maintained by the Application Area for the benefit of its clients i.e. the LHC experiments • to have the ROOT activity at CERN fully integrated in the LCG organization (planning, milestones, reviews, resources, etc.), • to ease the long-term maintenance and evolution of a single set of software products. • Details of the merge have been discussed in the AF LCG Applications Area Plans

9. SEAL and ROOT Merge (2) • The merge consists of two parts: • merge of the development teams into a single team • evolution of the software products into a single set of core software libraries • (1) happened immediately (Rene Brun as project leader) • (2) was originally foreseen to be completed within a very aggressive timescale but this has now been adapted to allow more time for the experiments to migrate their software • Experiments [ATLAS, CMS and LHCb] requested to continue the maintenance of the SEAL libraries • SEAL work package added to the ROOT project LCG Applications Area Plans

10. Smooth transition for Experiments time/versions • Adiabatic changes towards the experiments • SEAL functionality will be maintained as long as the experiments require SEAL Expts S/W ROOT functionality LCG Applications Area Plans

11. Merging functionality (sorted by priority) LCG Applications Area Plans

12. Core libraries and services (ROOT) • Provides basic functionality needed by any application • Current work packages • BASE: Foundation and system classes, documentation and releases • DICT: Reflexion system, meta classes, CINT and Python interpreters • I/O: Basic I/O, Trees, queries • PROOF: parallel ROOT facility, xrootd • MATH: Mathematical libraries, histogramming, fitting • GUI: Graphical Uner interfaces and Object editors • GRAPHICS: 2-D and 3-D graphics • GEOM: Geometry system • SEAL: Maintenance of the existing SEAL packages LCG Applications Area Plans

13. ROOT Milestones (level 2) BAS1 Base 01/08/2005 New updated plugin manager  BAS2 Base 15/07/2005 A new edition of the ROOT Users Guide  BAS3 Base 15/07/2005 The ROOT bug tracking system moved to Savannah  DIC1 Dictionary 30/06/2005 Reflex and Cintex in the ROOT v5 development release DIC4 Dictionary 30/04/2006 CINT fully converted to Reflex DIC5 Dictionary 31/03/2006 PyRoot exclusively based on Reflex  PRF7 Proof 30/09/2005 Demonstration of new PROOF system at ROOT 2005  MAT1 Math 30/06/2005 First version of new MathCore library  MAT2 Math 31/08/2005 First version of the MathMore and random number generator library MAT3 Math 30/09/2005 New C++ Minuit in be released in ROOT MAT4 Math 31/12/2005 Finalization of the Fitting and Minimization interfaces MAT5 Math 31/12/2005 RootFit fully integrated (in CVS by July) MAT6 Math 31/12/2005 Evaluation of a prototype linear algebra package independent of TObject  GRA1 Graph 30/09/2005 Demonstrate the new GL viewer at the ROOT workshop. SEAL1 Seal 30/11/2005 Remove from new SEAL releases Reflex and Cintex packages. Bug fix releases will continue to include them. SEAL2 Seal 30/11/2005 Remove from new SEAL releases Minuit and MathCore. Bug fix releases will continue to include them SEAL3 Seal 31/12/2005 Remove from new SEAL releases Reflection, ReflectionBuilder and the Scripting packages (PyLCGDict and PyLCGDict2) LCG Applications Area Plans

14. Software Process Infrastructure (SPI) • The AA projects share a single development infrastructure provided by the SPI project • Crucial for fostering homogeneity and avoiding duplications • Users are the LCG AA projects, LHC experiments and others external projects • Provides a number of “services” • Services • External Software service • Software distribution service • Savannah portal • Documentation and web site LCG Applications Area Plans

15. SPI Milestones (level 2) SPI01 31/10/2005 Provide the tools for generating CMT and SCRAM configurations from a common generic configuration description based on XML description files. Be able to update the web and distribution's kits from the same description. SPI02 30/11/2005 Independent build and release of partitions of the LCG software SPI03 31/07/2005 Pacman caches for binaries and sources SPI04 31/10/2005 Automated distribution system in place, generate from XML description External Software. SPI07 30/11/2005 Import/export features available in the Savannah system. Provide mechanism to import/migrate external data into savannah. SPI08 31/12/2005 Savannah user forum setup, user and admin documentation available. SPI09 31/10/2005 QA reporting on savannah and test coverage available and documented. SPI10 30/09/2005 Testing tools updated to the latest versions on the AFS area and available to LCG projects and outside. SPI11 01/08/2005 Doxygen and LXR automated and documented for the LCG software and for the packages (ROOT, AIDA, CLHEP) SPI12 28/02/2006 Provide a web based "user discussion forum" service interfaced with Savannah. This new service should allow projects and experiments to easily setup and manage discussion subjects. LCG Applications Area Plans

16. Persistency Frameworks (POOL) • Domain of expertise in data persistency, data management, deployment in the Grid and (relational) databases in general • The POOL project delivers a number of “products” • POOL, RAL, COOL • Work packages • Object storage and references • Collections and Metadata • Database access and distribution • Catalog and Grid integration • Conditions Database LCG Applications Area Plans

17. POOL Milestones (level 2) POOL01 Storage 31/12/2005 Reflex production release (Reflex validation by experiments completed) POOL02 Storage 31/03/2006 Overall performance study and validation with expected requirements RAL03 Ral 31/08/2005 RAL validation release (implementing the RAL review outcome and separate packaging) RAL05 Ral 31/10/2005 RAL production release RAL06 Ral 30/11/2005 Production release of proxy-cache access plugin CAT02 Catalog 31/08/2005 Production release of LCG catalog adaptors (level 2) CDB01 CondDB 30/04/2005 First COOL production release CDB08 CondDB 15/10/2005 COOL 1.4.0 release with bulk insertion and retrieval in different "channels" within the same "folder" CDB09 CondDB 15/10/2005 COOL passes all tier 0 validation tests CDB12 CondDB 31/12/2005 First prototypes of API and command line tools for data extraction and cross-population between COOL databases LCG Applications Area Plans

18. Simulation Project Leader Experiment Validation Geant4 Project Fluka Project MC4LHC Garfield Physics Validation GeneratorServices Framework Geant4 Fluka WP WP WP WP WP WP WP WP WP WP WP WP WP WP WP Simulation Project • Wide range of activities in the domain of expertise of event generators and detector simulation • Coordination effort to validate the physics of the simulations LCG Applications Area Plans

19. Simulation Milestones (level 2) SF530 Framework 30/09/2005 Flugg application to one of the LHC calorimeter test-beam simulation SF537 Framework 15/12/2005 Introduction of modularization of GDML files SF538 Framework 15/12/2005 Proposal for universal handling of Monte Carlo truth SF539 Framework 15/12/2005 Evaluation of VirtualMC for usage in Geant4-Fluka test-beam validation studies SF608 Framework 30/06/2006 Refinement to GDMLSchema to support user extensions of elements SF609 Framework 30/06/2006 Completed evaluation of VMC for use in physics validation SF613 Framework 31/10/2006 First release of a common framework for MCTruth handling SF615 Framework 15/12/2006 CPPGDML code revision and re-engineering to allow easier extensibility VD522 Validation 30/09/2005 3rd simple benchmark for physics validation completed VD524 Validation 30/09/2005 Validation of Fluka against one LHC calorimeter test-beam VD532 Validation 31/10/2005 First results of ATLAS combined and 2004 test-beams data comparisons VD612 Validation 31/10/2006 Validation of shower parameterization packages completed G4523 Geant4 30/09/2005 Development release G4531 Geant4 31/10/2005 Improved regression suite for release validation and testing infrastructure. G4542 Geant4 20/12/2005 New public scheduled release including positron annihilation and geometry voxelisation improvements. LCG Applications Area Plans

20. Simulation Milestones (level2) G4605 Geant4 31/03/2006 Development release including new tool for overlap detection at geometry construction and extensions to QGS. G4606 Geant4 31/05/2006 Development release. Including new features for parallel navigation enabling scoring charged particles at arbitrary locations G4611 Geant4 30/09/2006 Geant4 development release. Including developments including redesign of Binary Cascade's field transitions, additional benchmarking for radiation … G4614 Geant4 01/12/2006 Geant4 development release. Including surface tolerances tuned to model geometry size G4701 Geant4 01/06/2007 Geant4 development release. Including refined models for EM interactions of exotic particles, first implementation of tessellated BREP solids GF540 Garfiled 15/12/2005 Interface with a new version of Magboltz GF541 Garfiled 15/12/2005 Interface with a new version of Heed GS535 Genser 15/12/2005 Generator level production framework: production quality release GS602 Genser 31/03/2006 Introduction of MCDB Grid certificates and Management of large files GS616 Genser 15/12/2006 Fully operational LCG Generator production centre integrated in the grid-middleware GS701 Genser 30/06/2007 Generator level validation framework production version GS702 Genser 30/09/2007 Pythia 8. Release of production version LCG Applications Area Plans

21. Resources LCG Applications Area Plans

22. Conclusions • Planning is continuous activity. Publishing the plan document is not sufficient • Tasks, resources, milestones need to be revised often • Tracking progress must be done internally and externally • Plan revisions/updates • Quarterly updates as part of the LCG quarterly report • Annual updates per project also foreseen • Annual/18 months updates in “AA internal reviews” • Detailed planning (e.g. scheduling bug fix releases, configuration changes, etc.) discussed and agreed at Architects Forum meetings • I would like to thank all the project leaders and all the people that contributed in the preparation of the AA plan. LCG Applications Area Plans