1 / 27

SEAL: Core Libraries and Services Project

SEAL: Core Libraries and Services Project. CERN/IT After-C5 Meeting 6 June 2003 P. Mato / CERN. Contents. History: The blueprint RTAG Project Aims Current Work Areas: Foundation Libraries Math Libraries Plugin Management LCG Dictionary Scripting services Software Process

nayef
Download Presentation

SEAL: Core Libraries and Services Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SEAL: Core Libraries and Services Project CERN/IT After-C5 Meeting 6 June 2003 P. Mato / CERN

  2. Contents • History: The blueprint RTAG • Project Aims • Current Work Areas: • Foundation Libraries • Math Libraries • Plugin Management • LCG Dictionary • Scripting services • Software Process • Project Status and Summary SEAL - Core Libraries and Services Project

  3. The Blueprint RTAG • The mandate of the RTAG was to define the architectural ‘blueprint’ for LCG applications: • Define the main architectural domains (‘collaborating frameworks’) of LHC experiments and identify their principal components. • Define the architectural relationships between these ‘frameworks’ and components, including Grid aspects, identify the main requirements for their inter-communication, and suggest possible first implementations. • Identify the high-level milestones for each domain and provide a first estimate of the effort needed. • Derive a set of requirements for the LCG • Any piece of software developed by any LCG common project must conform to a coherent overall architectural vision • The end goal is the integration of LCG and non-LCG software to build coherent applications • The blueprint will be established in terms of a set of requirements, suggested approaches and guidelines, and recommendations SEAL - Core Libraries and Services Project

  4. Architecture requirements • Long lifetime: technology evolution • Languages: C++ today; allow multi-language and evolution • Distributed applications • TGV and airplane work • Modularity of components • Component communication via public interfaces • Interchangeability of implementations • Integration into coherent framework • Design for end-user’s convenience more than the developer’s • Re-use existing implementations • Software quality at least as good as any LHC experiment • Meet performance, quality requirements of trigger/DAQ software • Platforms • Trigger/DAQ environment SEAL - Core Libraries and Services Project

  5. Applications Reconstruction Framework Visualization Framework Simulation Framework OtherFrameworks . . . Basic Framework Foundation Libraries Optional Libraries Architectural Elements • Interface model • Abstract interfaces, versioning, guidelines,… • Component model • Communication via public interfaces (no hidden channels) • Plug-ins (run-time loading) • Life-time management (reference counting) • Configuration • Design guidelines • Dependencies • Exception handling • Interface to external components Software Structure SEAL - Core Libraries and Services Project

  6. EDG API PVSS XML Database GUI GUI Python shell math JPE PyROOT PyGaudi math Java Classes Root Classes Gaudi Framework Architectural Elements (2) • Object Dictionary • The ability to query a class about its internal structure (Introspection) • Essential for data browsing, rapid prototyping, persistency, etc. • Object Whiteboard • Uniform access to application-defined objects (equivalent to the Gaudi transient stores) • Component Bus • Integration of components providing a wide variety of functionality SEAL - Core Libraries and Services Project

  7. The Blueprint RTAG Recommendations • RTAG establishes a user/provider relationship between LCG software an ROOT • LGC software will not be based on ROOT, it will use ROOT where adequate • Start common project on core tools and services SEAL Project • Start common project on physics interfaces • Start RTAG on analysis, including distributed aspects • Tool/technology recommendations • CLHEP, CINT, Python, Qt, AIDA, … • Develop a clear process for adopting third party software SEAL - Core Libraries and Services Project

  8. SEAL Overview • SEAL aims to • Provide the software infrastructure, basic frameworks, libraries and tools that are common among the LHC experiments • Select, integrate, develop and support foundation and utility class libraries • Develop a coherent set of basic framework services to facilitate the integration of LCG and non - LCG software • Scope • Foundation Class Libraries • Basic types (STL, Boost, CLHEP, …), utility libraries, system isolation libraries, domain specific foundation libraries • Basic Framework Services • Component model, reflection, plugin management, incident (event) management, distributed computing, grid, scripting SEAL - Core Libraries and Services Project

  9. Domain Coverage SEAL - Core Libraries and Services Project

  10. Assumptions, constraints, risks • Do not re-invent the wheel • Most of the core software to be delivered by SEAL exists - more or less - in experiments’ core software • We will re-use as much as possible existing software • Most of the work will be in re- packaging existing pieces of software • If wheel squeaks… • Develop / adapt / generalize in order to achieve the necessary level of coherency and conformance to the architectural vision already established • Adopt a Seal • In order to use SEAL, projects will need to replace their own software elements with SEAL functionally equivalent ones. This will certainly imply some period of instability for the experiment applications SEAL - Core Libraries and Services Project

  11. Project Work Packages New work package added to the project by incorporating the MathLib project (F. James et al.) This work package will probably be split between “Foundation and Utility Libraries” and “Basic Framework services” later Not yet started working in this area SEAL - Core Libraries and Services Project

  12. SEAL Versions Road Map Released 14&26/02/03 Released 04/04/03 Released 23/05/03 SEAL - Core Libraries and Services Project

  13. 1. Foundation and Utility Libraries • Inventory of existing utility classes • Support for Boost library • Open source utility library (parts of which are destined to be included in the following C++ standard) • Participation to CLHEP project • Adopted CMS classlib • A set of foundation and utility classes mainly used in Iguana (CMS) • Development of SealUtil and SealKernel packages • The idea is to develop SEAL utility and system isolation library complementary to Boost, CLHEP and STL • Guidelines for selecting external libraries • Set of criteria to define SEAL - Core Libraries and Services Project

  14. 2. Math Libraries • GSL (Gnu Scientific Library) evaluation • How it compares with NagC • Automatic testing of GSL (with cppUnit) as part of every SEAL release • Re-implementation of MINUIT in C++ • Prototype available in current SEAL release • Migrad and Minos available • The numerical results of the two prototypes compared to the Fortran version. Compatible within the errors. • Other libraries • MathLibs/GSLAlgebra. Prototype wrapper around GSL. Not complete but restricted to Minuit needs. • Provide to experiments with math and statistics libraries to be used in analysis, reconstruction, simulation. • GSL support, … SEAL - Core Libraries and Services Project

  15. 3. Component Model and Plug-in Manager • Plug-in Management • Service in charge of managing, querying, [un]loading plug-ins • Implementation in two levels • Lower level (no framework constraints) • Ability to create “module” libraries that contain “plug-ins” • Ability to instantiate concrete implementations knowing the plug-in category and the concrete type • Caching information about what modules contain what plugins and their categories • Plugin instances management (assuming a given framework model) • Locate other plug-ins and manage their lifetime • Framework base classes to obey plug-in/framework model SEAL - Core Libraries and Services Project

  16. Object Factory Object Factory Plug-InCache Module Attached Plug-inDatabase Plug-InCache Module Plug-InCache Module Unattached Module Object Factory Module 3. Component Model and Plug-in Manager(2) • Howto’s available • Host modules in your project • Create a new plugin factory • Create a new plugin module library // Module Definition DEFINE_SEAL_MODULE(); DEFINE_SEAL_PLUGIN(SomeFactory, MyObject); // Usage by a client BaseObject* p; p = SomeFactory::get()->create(“MyObject”); SEAL - Core Libraries and Services Project

  17. 4. Object Dictionary • Adding “introspection/reflection” capabilities to C++ • Reflection packages • The Dictionary/Reflection package provides the capability to introspect and interact with any C++ object at run-time • The Dictionary/ReflectionBuilder package is the “write” interface of the reflection information. Typically generated code uses this interface to built the reflection information at run-time • Dictionary generation • Main goal: Full support of C++ without any class instrumentation • DictionaryGenerator package provides the command to do it • Usages so far • Object persistency (POOL) • Interactivity and scripting (Python) SEAL - Core Libraries and Services Project

  18. .xml selection file lcgdict .xml gccxml par ser filter + exten gendict .h _dict.cpp .so make .h .h (python script) .h #include files 4. Object Dictionary (2) • Dictionary generation from header files • Uses gccxml (0.4.0) to parse header-files (extension to gcc-compiler), generates intermediate XML file with dictionary information • XML file parsed by a python script which generates dictionary building C++ code (for “selected classes”) • Compiled to shared library and loaded at run-time to create dictionary in memory SEAL - Core Libraries and Services Project

  19. Other Clients: (python, GUI, etc.) LCG Object Dictionary: Usage . adl .h .xml Population ROOTCINT GCC - XML ADL/GOD CINT generated Dict generating code code in Conversion (1) LCG CINT Dict gateway LCG to CINT Dictionary Streamer Dict ROOT I/O (2) out Reflection SEAL - Core Libraries and Services Project

  20. 5. Basic Framework Services • Developing the basic framework • Component model definition • component identification, configuration, lifetime strategy, interface model, etc. • Started to review existing designs (Gaudi, Cobra, Iguana,…) • Basic Component Model • Set of “base classes” and interfaces that developers will use to build their components • Developing Basic Common Services • Started design and implementation for the message reporting service • Other services like component configuration, “event” management, object white board, etc. will continue soon after SEAL - Core Libraries and Services Project

  21. 5. Basic Framework Services (2) • The current ideas • Give to the “users” (framework developers) a number pre-defined service base classes (from simple to more complex) • Developers should “model” their services using one of the proposed base classes • Standard functionality (instantiation, configuration, lifetime management, etc.) should be provided by these bases classes Client NameSvc Context Service PropMgr MyService AServLet BServLet SEAL - Core Libraries and Services Project

  22. 6. Scripting Services • Scripting needed for Interactivity, Configuration, Rapid prototyping, Integration, etc. • GOAL: Bring Python and C++ (CINT) at the same level of functionality • Interoperability between them. The end-user choice. • Define guidelines for developing Python bindings • Evaluate existing options: SWIG, Boost.Python, SIP, and raw Python C-API • Study how each technology handles a number of predefined cases • Examples: Method overloading, inheritance across boundaries, templated classes and methods, natural mapping of Python constructs, etc. • Study interoperability between binding technologies SEAL - Core Libraries and Services Project

  23. 6. Scripting Services (2) • PyROOT: Python bindings for ROOT (former RootPython) • PyROOT is a Python extension module that allows the user to interact with any ROOT class • Done generically using the ROOT dictionary. No need to generate any Python wrapper code to include new ROOT classes • PyLCGDict: Python extension module • Started to work on the LCG Dictionary bindings • Basically repeating the work of PyROOT using the LCG dictionary instead SEAL - Core Libraries and Services Project

  24. 8. Education/Documentation • Main activities and tasks • Produce documentation • Produce training material (tutorials) • Help incorporating SEAL components into LCG projects and experiment frameworks • Existing documentation • Very limited for the time being • Produced a number of HowTo pages for the released elements • Plugin Manager, Dictionary generation, PyROOT, etc. • Helping POOL to incorporate released components • We have a team member working for both POOL and SEAL • Will be preparing tutorials for June release • Time of possible adoption by experiments SEAL - Core Libraries and Services Project

  25. Software Process • SPI provided infrastructure • Savannah Web portal, CVS repository, development tools (SCRAM, Doxygen, CppUnit, etc.) and services (external tool repository, etc.) • Slowly getting the good dynamics and practices • Efforts to “standardize” tool usage and conventions • Avoiding divergences between LCG projects • Adaptation of tools to “LCG proper” working models • Release procedures • Developing SEAL proper procedures • Tagging, building, testing, releasing, documentation generation, announcing, etc. • Automation of the process is essential (rotating release manager role) • Platforms • Single release platform for the moment: Linux RedHat 7.3/gcc-3.2 SEAL - Core Libraries and Services Project

  26. Status and Summary • The SEAL project started last November as a result of the “blueprint” RTAG recommendations • SEAL has had three main releases: 0.1.0, 0.2.0, 0.3.0 since the beginning of the year at the scheduled times • The main emphasis has been to support POOL (Dictionary, Plugin Management, Foundation classes, etc.) • The functionality available is not yet enormous but its development has help us • To get experience on new tools and procedures • Build up the development team • No technical problem encountered so far • Next scheduled release • Basic functionality completed and ready to be adopted by experiments at end-June SEAL - Core Libraries and Services Project

  27. http://seal.cern.ch/ SEAL - Core Libraries and Services Project

More Related