FSR – frustrating summary records

1. FSR – frustrating summary records file ^ Update R. Lambert Core Soft, 10th October 2012

2. Possible Solution • Fixing any of these will fix FSRs for good • Get sparse trees in Root to work properly • Have basket size and optimization tuneable tree-by-tree • Markus realized this is possible in LHCb code • Available in head or Online/RootCnv … let’s try it … 10,000 FSRs Sparse Trees in TES Sparse Trees Persisted ROOT Tree used ROOT can’t handle sparse trees Core Soft, 10th October 2012

3. Two Parameters • Buffer: • Reserved memory for each leaf in the Tree • Basket: • Supposed sizes ofall the leaves Core Soft, 10th October 2012

4. Test Cases A B Real Data 13475 FSRs, 4 levels MC Filtered 40 FSRs, 7 levels Lxplus 135 Only me using it My code on local disk LHCb code on cvmfs Write to local disk Read from Castor Verified stability of CPU/Vmem CPU : 10% Vmem: 0.5% File size: 0% Core Soft, 10th October 2012

5. Buffer A B Core Soft, 10th October 2012

6. Buffer Original RootCnv A B Core Soft, 10th October 2012

7. Buffer Markus/Ivan optimized A B Core Soft, 10th October 2012

8. Buffer Suggested optimum A B Core Soft, 10th October 2012

9. Basket Completelynot relevant A B Core Soft, 10th October 2012

10. Results • Only minor improvements possible! SetupProjectDaVinci v32r2p1 time gaudirun.py ~rlambert/public/forMarkus/fsrMemLeak/options.py #100 files with one event each, but 13475 FSRs in total Core Soft, 10th October 2012

11. Summary • Re-optimizing for FSRs not as good as we hoped • Writing *any* FSRs causes a leap in Vmem and CPU time • Need to keep merging, such that we never have 10k FSRs • Even 40 FSRs are a problem if buried under many levels • Again we can turn to these other options, but they are all very difficult … • Automatic merging • Event-like FSRs • Flattening (Three possible directions) • Abandon trees altogether Core Soft, 10th October 2012

12. End • Backups are often required Core Soft, 10th October 2012

13. Discussion Core Soft, 10th October 2012