410 likes | 493 Views
Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better. (S. Beckett). CPass0/CPass1 on LHC12e/d/c Updated at 10:00 on 20/08. C. Zampolli. LHC12f. Summary table – on 20/08 at ~ 10:00 LHC12f. 25 in logbook
E N D
Ever tried. Ever failed. No matter. • Try Again. Fail again. Fail better. • (S. Beckett) CPass0/CPass1 on LHC12e/d/cUpdated at 10:00 on 20/08 C. Zampolli
LHC12f C. Zampolli
Summary table – on20/08 at ~10:00LHC12f • 25 in logbook • Filters used: LHC12f, PHYSICS, Good Run, GRP ok at least one of [SDD, TPC, TRD, TOF, T0] • CPass0, completed: • Snapshot: 26 (run 186687 – 2 min - marked as bad later) • Reco+CalibTrain: 26 • Merging+OCDB: 25 (186845 still running in the reco), 23 needed, 19 ok • CPass1, completed: • Snapshot: 19 • Reco+CalibTrain: 19 • Merging+OCDB: 19 C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12f • COSMICS: 0 failure expected • EMCAL/PHOS/MUON: 2 failure expected • No triggers: 0 failure expected (too short run) • EE/EV/Expired: 0 memory issue during the merging (under investigation) • Running: 0 • Others (detectors): 4 (186855 186816 186694 186687 • Successful: 19 • 19/(19+4) = 82.6% success rate C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12f 2 min 12 min 6 min 7 min All failures due to too short runs C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12f C. Zampolli
Summary table – on20/08 at ~10:00CPass1 – LHC12f • Of the 19 successful runs: • 19 at CPass1 reco+CalibTrain • 19 at CPass1 merging+OCDB C. Zampolli
LHC12e C. Zampolli
Summary table – on20/08 at ~10:00LHC12e • 27 in logbook • Filters used: LHC12e, PHYSICS, Good Run, GRP ok at least one of [SDD, TPC, TRD, TOF, T0] • CPass0, completed: • Snapshot: 27 • Reco+CalibTrain: 27 • Merging+OCDB: 27, 21 useful, 11 ok • CPass1, completed: • Snapshot: 11 • Reco+CalibTrain: 11 • Merging+OCDB: 11 C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12e • COSMICS: 0 failure expected • EMCAL/PHOS/MUON: 6 failure expected • No triggers: 0 failure expected (too short run) • EE/EV/Expired: 0 memory issue during the merging (under investigation) • Running: 0 • Others (detectors): 10 • Successful: 11 • 11/(11+10) = 52.4% success rate C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12e • TRD: • (*) suffered from missing class (CSPI8WU-S-NOPF-ALL) in the configuration during data taking • Fixed manually using CINT8WU-S-NOPF-ALL • Cpass0/1 should be re-run • (**) suffered from statistics – 186459 has CSPI8WU-S-NOPF-ALL but with zero triggers) • T0 suffers from high background, but limits will be increased • Re-running will be ok (but CPass1 should be triggered manually if Rev < Rev-23 will be used) C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12e C. Zampolli
Summary table – on20/08 at ~10:00CPass1 – LHC12e • Of the 11 successful runs: • 11 at CPass1 reco+CalibTrain • 11 at CPass1 merging+OCDB C. Zampolli
Actions • CPass0 completed on the available runs • 10 runs failed • 2 T0 (1 in common with TRD) • CPass1 can be triggered manually at any time • If re-running everything with Rev > Rev-23 (the next to come), everything should be ok, otherwise CPass0 will fail again, and CPass1 will be needed to be triggered manually • 9failed in TRD (1 in common with T0) • 5 runs had not the right class in the configuration • Fixed manually, waiting for OCDB update to re-run • 4 runs have too little statistics • CPass1 completed on the available runs • In summary, 6 runs can be recovered C. Zampolli
LHC12d C. Zampolli
Summary table – on20/08 at ~10:00LHC12d • 224 in logbook • Filters used: LHC12d, PHYSICS, Good Run, GRP ok at least one of [SDD, TPC, TRD, TOF, T0] • CPass0 completed: • Snapshot: 220 • Reco+CalibTrain: 220 • Merging+OCDB: 220, 176 needed, 147 ok • CPass1 completed: • Snapshot: 148 (1 more than CPass0, triggered manually after CPass0) • Reco+CalibTrain: 148 • Merging+OCDB: 148, 148 needed C. Zampolli
Difference between logbook and snapshot in MonALISA • In logbook, but not in MonALISA: • 184370 (EMCAL), 184645 (EMCAL), 185345 (ACORDE trigger), 185347 (ACORDE trigger), 185467 still in the migration process, checking with offline • In MonALISA but not in the logbook: • 185190 (short run, the quality flag was changed) C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12d • COSMICS: 9 failure expected • EMCAL/PHOS/MUON: 33 failure expected • No triggers: 2 failure expected (too short run) • EE/EV/Expired: 1 memory issue during the merging, but then merged manually • Running: 0 • Others (detectors): 28 • Successful: 147 • 147/(147+28+1) = 83.5% success rate C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12d Also TRD 16 recovered rerunning with looser constraints for validation (run 185460 not retried, since it failed anyway in TRD) C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12d Hardware problem, fixed now C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12d C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12d Also TPC Merged manually C. Zampolli
Summary table – on20/08 at ~10:00CPass1 – LHC12d • Of the 147 successful runs: • 148 at CPass1 reco+CalibTrain • 1 more than CPass0 since CPass0 was merged manually and the objects were uploaded manually in the OCDB (184673) • 148 at CPass1 merging+OCDB… • …of which 147 successful (ignore the red TPC color)… • ...1 failed in TRD (184145)… • Different statistics for CPass0 and CPass1 • 480/480 chunks at CPass0 • 472/480 chunks at CPass1 C. Zampolli
TRD issue • Due to a problem in the TRD reconstruction, some wrong OCDB entries were produced at CPass0; it is not possible to get the correct ones without re-running CPass0 • Some manual OCDB update is needed (after LHC12d is fully processed, ongoing for completed runs) • Then CPass0/CPass1 should be re-run with a Rev > Rev-18 • Will the failed runs be recovered? Waiting for experts’ reply C. Zampolli
Actions • CPass0completed • 20 runs failed at CPass0 due to T0 hardware problems • CPass1 should be triggered manually for these runs • To be done after reprocessing, since now it would be useless (they all contain TRD) • 8 runs failed in TRD • TRD needs LHC12d reprocessing (only for the runs it was in) • will these 8 runs be recovered, or the failure reason is something that won't be fixed when re-running? • run 184673 failed in CPass0 merging (EV) and had CPass0 entries uploaded produced manually by Raphaelle, and uploaded in the OCDB • CPass1 run, everything seems ok C. Zampolli
Actions – II • CPass1 completed • 1 run failed in TRD due to lower statistics at CPass1 reconstruction • should we try to recover it? will the TRD people fix it manually before VPass? Probably not needed, since we should re-run everything for TRD anyway • In summary, we are waiting to re-run for TRD C. Zampolli
LHC12c C. Zampolli
Summary table – on20/08 at ~10:00LHC12c • 205 in logbook • Filters used: LHC12c, PHYSICS, Good Run, GRP ok at least one of [SDD, TPC, TRD, TOF, T0] • Do not coincide with those in MonALISA, since runs were queued manually for CPass0 • CPass0 completed: • Snapshot: 208, 1 should be ignored (179444) • Reco+CalibTrain: 207 • Merging+OCDB: 207, 109 needed, 93 ok • CPass1 completed: • Snapshot: 93 • Reco+CalibTrain: 93 • Merging+OCDB: 93 C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12c • COSMICS: 37 failure expected • EMCAL/PHOS/MUON: 58 failure expected • No triggers: 3 failure expected (too short, or not the right trigger configuration) • EE/EV/Expired: 0 • Others (detectors): 16 • Successful: 93 • 93/(93+16) = 85.3% success rate C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12c C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12c C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12c C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12c C. Zampolli
Summary table – on20/08 at ~10:00CPass0 – LHC12c (*) Low statistics, recoverable (*) Low statistics, not recoverable (**) No SSD/SDD number of contributors to Vertex Track = 0, TRD calibration failing, TRD fix in place; what about TPC? C. Zampolli
Summary table – on20/08 at ~10:00CPass1 – LHC12c • Of the 93 successful runs: • 93 at CPass1 reco+CalibTrain • 93 at CPass1 merging+OCDB… • …of which 84 successful in CPass1 (ignore the red TPC color)… • …and 9 failed in T0, but are MUON runs – they should have not gone through (different AliRoot, some changes in T0) • As soon as CPass1 is completed, 1 week of time will be given for manual update. If too little (QM, holidays), we’ll increase it. Then, Vpass should start C. Zampolli
Actions • CPass0 completed; • 9 runs failed in TPC and TRD • TRD failed due to missing SDD/SSD; • what about TPC? • TRD provided a code fix • would TPC try to recover these runs? • if both TPC and TRD can recover, should we wait for this and then run again CPass0/CPass1? • 7 runs failed in TRD due to low statistics • TRD can recover them manually, but no CPass1 would be run after those • how will the other detectors mark these runs? • TOF, T0 bad • Mean Vertex good • TRP? TRD? • CPass1 completed on the available runs • In summary, we need to know whether the 9 runs that failed in TRD+TPC should be reprocessed need a statement from TPC C. Zampolli
Further comments C. Zampolli
Interdependencies • Under discussion: does EMCAL runs need calibration triggers? (PHOS does not) • Seems not! C. Zampolli
Further issues • Some reconstruction jobs fail with bad_alloc under investigation • Grid tests with gdb ongoing not many information retrievable, the jobs ran successfully • Valgrind test ongoing did not show anything significant • Trying with Rev-21 on LHC12c, LHC12e • Many errors, but FPE, not bad_alloc • stack trace available • I could not reproduce the problem, still investigating C. Zampolli
PPass • LHC12a and LHC12b Vpass validated ready for Ppass • A patched Rev-16 was created to fix the TRD QA issue to be used to run Ppass • LHC12a completed, waiting for QA feedback • LHC12b completed, waiting for QA feedback C. Zampolli
Calibration of old data • GRP/CTP/Aliases entries to be created, after defining the classes to be used for the reconstruction • Might be needed to apply some downscale • min(max(nevents/10,30000),nevents)/nevents, but we need to define nevents C. Zampolli