1 / 11

Data size and IO performance report

This report by Ilija Vukotic discusses MC event sizes, streams sizes, categories, IO performance, and possibilities for improvement in ATLAS Software. It includes detailed information on data sizes, event sizes, and recommendations for enhancements in performance and efficiency.

sylvester-evans
Download Presentation

Data size and IO performance report

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data size and IO performance report ATLAS Software & Computing Workshop

  2. Data size • MC http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/MCsizes.html • Streams http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/StreamsSizes.html • Categories http://athena-infoioperformance.web.cern.ch/athena-infoIOperformance/T0streams.html • IO performance • Possibilities for improvement Ilija Vukotic

  3. MC event size Ilija Vukotic

  4. MC event size *Compared to 22 Aug. 2010 Note: trigger size is not realistic as used trigger menu was not realistic. Problem rectified with addition of new aliases: Physics_default -> Physics_pp_v1 MC_loose_default -> MC_pp_v1_loose_mc_prescale MC_tight_default -> MC_pp_v1_tight_Mc_prescale (see bug report #74712). Ilija Vukotic

  5. Streams sizes • Only runs with 5k+ events • and good LBs • Shows: • AODs,ESDs and all DESDs • Events/total size/ev. size Ilija Vukotic

  6. Streams sizes ESD DESDM_EGAMMA AOD DESD_SGLMU DESD_SGLEL Ilija Vukotic

  7. Categories Ilija Vukotic

  8. Categories JetTauETMiss Muons L1Calo Egamma Details and list of collections in categories can be found at: https://spreadsheets.google.com/ccc?key=0AiPvgbRljNCodG5IY1BseHRHa2R5WjZoMGRnLXp0dHc&hl=en Ilija Vukotic

  9. IO performance Slow Bad CF KEEP IN MIND In real data TrackCollections are even larger ! Ilija Vukotic

  10. IO performance • Do we really need all of these? • No way to say if object is ever used. • What is so terrible about having a name attached to each object we store? Ilija Vukotic

  11. Possibilities for improvement • TrackCollection/TrackParticle container • Maybe 50kb in size and factor 2 in time • Few MB in memory • CaloShowerContainer • Probably can go down in size • Calo(Topo)TowerContainer • Gets recreated at read time • Must be possible to optimize further • TRT_Drift_Circle_Container • Must be faster and smaller My estimate: In total between 50 and 200kB/ev can be shaved in converters Should be possible >100ms/ev (currently at 480 ms/ev) Other options: Object removal. Cuts tuning Ilija Vukotic

More Related