1 / 10

Automated Certification System for Training Tasks in PWG1

Proposal for developing an automated certification system for PWG1 training tasks to ensure code stability and efficiency, with detailed guidelines for testing, evaluation, and certification of tasks. The system includes macros and scripts to facilitate task configuration, execution, and analysis, aiming to streamline the certification process. The system also emphasizes user involvement and provides clear decision criteria for task certification.

flynnd
Download Presentation

Automated Certification System for Training Tasks in PWG1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Marian Ivanov PWG1 train structure - proposal

  2. PWG1 train • PWG1/QA train tasks to be run (also) together with reconstruction/calibration • PWG1 train consist from many tasks written by many authors • Probability to crash • Logical or (saturation at 100%) • Memory and CPU consumption • Additive • Log and output size ... • Automatic schema to guarantee the stability of the code has to be developed

  3. PWG1 train • Objective criteria to certificate (or do not certificate) tasks • User should be able to run the same certification code as train administrator who made automatic decision • To be ready for tags • Test to be performed independently for groups of tasks • AddPWG1<group>.C • Similar schema already used for other PWGxxx • I tried to make the schema automatic • Ideally part of nightly builds • In case of problems authors automatically notified as in SHUTTLE

  4. PWG1 Train • Macros and scripts to run PWG1 train in $ALICE_ROOT/PWG1/PWG1macros: • runPWG1Train.C - Steering macro • Load configuration for all tasks Add<PWG1>*.C • Macro does not configure tasks 0 default arguments has to • Run train • The same macro used also for tests • Add<PWG1>*.C - Macros to configure user tasks • All macros should have default parameters - otherwise they are ignored • Task can get run number information, trigger information during analysis from ESDs • To add: • Name and e-mail of responsible to be in Add*.C • Syntax e.g : Responsible: marian.ivanov@cer.ch

  5. Naming convention • Strict naming convention to be defined • Suggestion: • AddPWGX<taskGroupName>.C • OutputPWGX<taskGroupName>.root • One output per taskGroup • Not always case • AnalyzePWGX<taskGroupName>.C • Macros to extract typical QA values

  6. Tasks certification • User can commit to $ALICE_ROOT/PWG1/PWG1 macros once fulfill previous criteria. • Not all of the macros will be used for production • Only certified macros will be copied to the production directory (not on SVN) • Test are performed per macro Add*.C with given input esd.list. • The esd.list used should be announced to users before tagging • Decision criteria: • No crash • Reasonable CPU per event • Reasonable increase of VM per event • Reasonable size of log files • Web interface preferred to browse the results of tests • Nightly/weekly builds

  7. Certification Scripts • getCertificate.sh - User script to test task • - Input: • 1. Input name of macro • 2. esd list • - Output: • 1. out.log • 2. summary.log • makeSummary.sh - Make summary information • getCertificateAll.sh - Pseudo code for train administrator • 1. Submit jobs for all user macros Add*.C • 2. Wait • 3. Make summary logs • Part to be implemented • 4. Copy certified Add*.C to certified directory • 5. Write e-mail to responsible in case of problems

  8. Summary logs per group. Examples (Trunk20.03.2010) TPC task to be formally certified Actual certification – analyze function to be implemented Email to be sent

  9. Summary info examples (Trunk20.03.2010) • Summary- Increase of memory consumption per event (Mby) • Ideally 0 • Summary- Increase of memory consumption per event • Summary- Increase of time consumption per event • Common shift due IO • + one outlyer

  10. OCDB access in PWG1 tasks(Trunk20.03.2010) • For PWG1 tasks the OCDB access should be enabled • ConfigOCDB.C(runNumber) to set in steering macro • Magnetic field setting, initialization of the GeoManager, and alignment • Set default storage/Run number • User tasks accessing the OCDB should just make statistic of problem – do not crash

More Related