Status & development of the HYDRA framework

1. Status & development of the HYDRA framework Jochen Markert HADES collaboration meeting 2010, GSI

2. Status after p+Nb beam • Future experiments: • Heavy ion collisions with high track multiplicity • Upgrade of HADES: • Replace TOFino by RPC detector • New DAQ • New hld Event format • ~ 50 times higher data volume HADES collaboration meeting 2010, GSI

3. Consequences • Replacement of TOFino by RPC: • Unpacker + Calibrater + Hit -/Clusterfinder + Digitizer of RPC to added • Since RPC does not simply replace TOFino a new integration scheme for tracking is needed • Change of hld format: • New Unpackers for all detectors needed • Track multiplicity in central heavy ion collisions: • increase of combinatorics, computation time, tracking ghosts • Decrease of tracking efficiency and quality • Increased data volume: • New strategy for DST production and analysis needed HADES collaboration meeting 2010, GSI

4. Upgrade strategy I (source code management) • Lots of code changes and clean up of 10 years of history to be done • CVS repository is closed • move to SVN repository for better management • Permissions for developers managed by database • Code can be checked out / commited to the web server • hydra (https://subversion.gsi.de/hades/hydra) • containing all history up hydra v8_21. • will not be touched besides important bug fixes. • should be used for analysis of old data (up to sep08). • hydraTrans (https://subversion.gsi.de/hades/hydraTrans). • For the transition period to the final hydra frame work. • development version. All changes of the code will be committed here. • not backward compatible to read old DST ROOT files • Should be used for simulations for the heavy ion runs • hydraII (https://subversion.gsi.de/hades/hydraII). • final repository used for the analysis of the upcoming experiments 2010 and later. HADES collaboration meeting 2010, GSI

5. new Tasks for MetaMatch, Spline and RK (later labeled "new"). • new pid/pair to have different container classes and therefore a new Filler + Sorter/Cleaner. • Old analysis with ShowerTofino will not be supported by the new data structure. HADES collaboration meeting 2010, GSI

6. Task assignment • Vladimir, Olga: Tracking • Anar: adaption of Spline/RK • Ilse : ORACLE/RuntimeDB, Unpacker integration, START detector • Jochen: common framework, replacement of old PID/PAIR • Gosia, Tatyana: Analyis development • Detector libs + Unpacker by detector groups HADES collaboration meeting 2010, GSI

7. Time line A bit delayed…but not too much HADES collaboration meeting 2010, GSI

8. Changes in hydraTrans • done • Remove lib: • alignment lib • mdctrackS lib • online lib • phyana lib (HPhysicsConstants moved to base/util) • showerTofino lib (remaining classes now in shower lib included) • remove GO4 , rename hadesgo4 to online lib (see Julian Thomas talk on Friday) • add new lib • startNew dir for development of new start detector (not included into Make) • oraNew (as above) • Particle (replace old pid/pairs lib) • new tracking code (not fully finished) • Integration of RPC (code for parameters to be done) • remove cross dependencies from libs • To be done • remove kickplane lib • Adopt dst lib • remove obsolete classes from all libs (partly done) • add pair framework to lib particle HADES collaboration meeting 2010, GSI

9. Where are we now? • Full simulation chain: • RPC detector integrated • Full DST chain: • RPC integrated • New MDC tracking code • Efficiency + purity results looking promising (see Vladimir’s talk and proposal) • Tracking speed strongly improved (2 events/s fully central Au+Au collisions) • New DST output format (see next transparency) • Preliminary Lepton analysis in Au+Au available (see Tatyana’s talk on Friday) • Missing: • Iterative track finding for off-vertex tracks • Global track fitting (Kalman filter etc.) • Speed optimization HADES collaboration meeting 2010, GSI

10. DST format considerations • Old HPidTrackCand objects are very large  Disk space • Many not needed data members … • Old theorem: “collect everything and decide at the end” not feasible anymore  too many combinations HADES collaboration meeting 2010, GSI

11. What has to be done … • Strategy (in Tracking, see. Vladimir‘s talk): • Remove as many ghosts from inner part of the MDCs before doing combinations with outer detectors • Use TOF hits in candidate search to reduce the number of combinations • Remove “non physical” candidates before going to fit procedures • Change output format: • Keep minimum number variables for most tasks • Allow extensions to the output objects for special purposes • Reduce variable types to appropriate value range • Use compression features of ROOT • Do not write RuntimeDatabase to output file • Do not write HGeantKine category to DST output in Simulation • Enable read of GEANT info with second ROOT source if needed HADES collaboration meeting 2010, GSI

12. Comparison of object sizes * ROOT output is compressed in addition (factor typical 0.6) HADES collaboration meeting 2010, GSI

13. Further reduction of data volume • Float_t variables need 4 bytes, but often the precision is not needed in the DST output • Use Float16_t (truncated) • In memory 4 bytes, normal Float_t precision • Truncated when written to disk • 30 % reduced data size on disk (> 50 TB disk space) • Do not store RuntimeDB in each ROOT file • ~ 2 TB disk space HADES collaboration meeting 2010, GSI

14. HParticleCand HParticleCand HParticleCand HParticleDebug HParticleDebug HParticleDebug HParticleCal HParticleCal HParticleCal Flexible extension of DST output … 1st object 2nd object 3rd object • For special tasks additional objects can be stored in the same order in the DST output • Objects can be easily accessed by index • Even T->Draw(….) commands will work since objects are stored exactly parallel • DST output can be setup for example like: • Do 10% of files with Calibration info for monitoring etc HADES collaboration meeting 2010, GSI

15. Compute requirements estimation • beam intensity of 1.5 * 107 ions per 5-sec spill. • 1% interaction target •  reaction rate of 3 * 104/sec. • LVL1 trigger set to select 25% of all reactions, • 10 kHz accepted LVL1 • 4.32 * 108 semi-central LVL1 events per day on disc (assuming 50% duty factor and dead time). • maximal possible trigger rate of 20 kHz will be filled with (scaled-down) peripheral reactions HADES collaboration meeting 2010, GSI

16. 30 days beam • Au+Au • Hld event size estimated from simulations 25kB • DST event size 9kB • Read speed 20MB/s HADES collaboration meeting 2010, GSI

17. CPU power • Real time: 3325 CPU cores needed •  unrealistic • Compromise: • 1500 CPU cores 3 month per year for DST • 1000 CPU cores full year for analysis • Situation now: • 400 CPUs + X  600 parallel jobs • Where to find additional 1100 CPU cores ? • What about disk space (> 400 TB missing)? HADES collaboration meeting 2010, GSI

18. Summary • Software already now in good shape • Still features missing … • GOOD MESSAGE: Hades can handle heavy ion collisions • What about the IT environment ? • Reliability of Lustre / batch farm • Missing resource: CPU + disk space • Man power …. HADES collaboration meeting 2010, GSI