Baselining COBRA for LHC operation

1. Baselining COBRAfor LHC operation Vincenzo Innocente CERN/PH new COBRA

2. Cobra status - History • Cobra started as a place to experiment and prototype framework mechanisms • Was expected to evolve and be re-engineered using the most adequate solution Instead: • Evolution of COBRA in last few years dominated by “operation” requirements • Objy ==> Pool • Support of production and data-challenges • backward compatibility • New features just piled-up (often as work-around) • No opportunity to fix very basic mistakes • Impossible to “abstract” commonalities and avoid unnecessary duplications • SimEvent,RecEvent,RawEvent,G3EventProxy • SimApplication,RecApplication • (P)RecDet, RecUnit and their variations new COBRA

3. Cobra Long-Standing Issues • Multiple event structures • Too many ways to achieve (almost) the same result • Multiple interfaces to the “event” • Integration with configurations, conditions and data management • Migration to foundation classes from IGUANA/SEAL • Build system • Core services (Geometry) are neither managed nor integrated into the framework • Chaotic I/O due to forced evolution of persistency new COBRA

4. Architecture Review • Have been at the top of CMS to-do list for a long time: • Café, LCG/AA Blueprint, Naples-workshop • Short-term urgency always deferred it • Combination of: • Fresh look at CMS framework (LPC) • New requirement from analysis (APROM) • Window of opportunity before Physics TDR new COBRA

5. Review Process Charged by PRS & CCS to: • Make a list of important features needed based on RunII, part of a draft document • Examine their presence in the CMS framework Extract a list of missing features (posted doc) • Produce a list of desirable changes to CMS framework • Present a possible work plan to implement the above (design doc by years’ end - i.e. next week) new COBRA

6. Responses from Collaboration • Following two weeks of work at FNAL • Initial presentation to a special JTB mtg (VI, Nov) • Main types of comments: • Not enough documentation in presentation Posted the first note and have an advanced draft of design note • Questions of the “why-now”, “why-do-we need-this” varieties. Addressed in Paris’ presentation • Specific technical questions for clarification Collected and will address each one new COBRA

7. Q&A • Detailed design doc will help address these and many additional ones • Will try to address few recurring ones: • Reco-on-Demand • Will be kept (in parallel with explicit pre-configured scheduling) • Detector Oriented Reco • No problem - in principle • Issues with details of implementation • Consistent configuration • Storage of results • Provenance tracking • Core framework mechanisms • Singletons and such: will be replaced by SEAL services • Access from ROOT and its implications • Requirements from PRS and APROM are clear: new COBRA

8. Required Usage Patterns: • “Bare Root“. The stored form of objects has to be sufficiently simple to allow their use without class libraries or additional code. • ROOT with a small set of libraries. This can use classes of type "basic elements". We want the classes of this second level to be defined by CMS. Analysis of data in this format shall not require access to external databases. • ORCA-Lite. Within ROOT or Framework with a medium set of libraries. This can use classes of type "1" and also of type "2". Analysis of data in this format shall not require access to external databases. • Full ORCA. Within the framework, using the full reconstruction program. This includes not only the software libraries but also includes full connectivity to any external resources. new COBRA

9. What Does Use of Root Look Like? • To figure out what the branch structure of the data should look like, we discussed what we want to use at the Root prompt (in case "1"). • Our example assumes a branch "ele" carrying electrons from one algorithm, and a branch "trk" carrying tracks from one algorithm. • Our example is to plot the pt of the track associated with each electron in the electron collection: t->Draw("trk.pt[ele.tk.id]") • Raw data is stored in a ROOT tree as well, but is not required to be transparent to Bare ROOT. new COBRA

10. What will change • One Event class (edm::Store) • Containing a flat list of event-data-products • One super-type of event-data-product (edm::Product) • Directly accessible from the event (ev.get(…) ) • Replace SimEvent substructs, RawData, Reconstructor • Fully specializable by “users” • Most common type (vector<T>, vector<T*>) provided by edm • Will replace RecCollection and its friends • One super-type of event-data-producers • Can produce multiple products • Invoked by the scheduler or (in ROD) as a result of a ev.get(…) • Replace EvSource, part of (P)RecDet, RecUnit • Extend RecRef to all Products • Simplified implementation • Builders will use Seal::Plugins and factories new COBRA

11. What will change (continue) • There will be One type of Application • Fully configured loading proper edm::producers • Persistency will use Pool (more than now) • COBRA persistency layer closer to native pool • Stream objects in Root trees • Try to modify pool to populate a tree not a grass-field… • Avoid updates (winter mode colder than in Minsk…) • Global Job data (metadata) will be streamed • Either in the same file (different root-tree) • Or in a SQLight file using Pool-RAL • To be studies together with DM • Data publishing (AttachRun) will be full responsibility of DM layer new COBRA

12. What will change (configuration) • Configuration will be “read” from an external source in form of ParamterSets • No built in default • ParameterSet will be identified uniquelly and globally • Producers shall be configured by the Builder (seal Plugin) using a ParameterSet • Producer are managed by the framework • Sub-Algorithms (today ConfigAlgo) will be constructed and owned by the parent Producer (or sub-algo) using a ParameterSet derived by its own • No specific super class for Sub-algo • No global/shared “ConfigAlgo” new COBRA