MC/Offline Planning

1. MC/Offline Planning M. Moulson Frascati, 21 March 2005 MC status Reconstruction coverage and data quality Running time estimates MC & reprocessing production plans

2. Offline/MC tasks for 2006 • Reconstruction of 2006 data (√s = 1000 MeV) • Data quality and database maintenance • Make sure all reconstructed runs are complete • Make sure bad runs flagged and reconstructed files deleted • Close holes in data reconstruction and DST coverage • DST file size problem = 1 week of scripting work • MC production for 2004-2005 • Reprocessing of various data sets • Most critical may be 2004 data with bad wire maps • Need to define other data sets to reprocess • Re-reconstruction of 2001/2002 MC sample • Still interested? 30 pb–1? More?

3. Reconstruction of 2004-2005 data Updated to reflect DB work 21 Mar 06 Runs 28700 (9 May 04) to 41902 (5 Dec 05) 1970 1930 1740 1000 1490 1530 1490 pb–1

4. Check of reconstruction coverage • Ensure that no reconstructed files or DSTs present for runs flagged bad • Review runs with no vote in run_logger: • 1828 runs/11.8 pb-1 generally due to DAQ crashes • 112 runs/11.6 pb-1 with integrated luminosity > 20 nb-1 • Check logbook for these runs and vote accordingly • Ensure reconstruction complete for each run and DBV

5. Incomplete reconstruction 2004-2005 • For each run and DBV: • Require all raw files have corresponding kpm, ksl, rpi, rad, clb, bha, and lsb files • Entire reconstruction for this DBV deleted if any files missing, together with any DSTs • Database record deleted, as if run were never reconstructed • Reconstruction pass with latest versions to close holes • Separate DST pass to close holes once we have: • Complete 2004-2005 reconstruction • Script modifications to handle big DSTs

6. Incomplete reconstruction 2001-2002 • More complicated! • Data have already been analyzed • bgg files were made from bha files - DB2 dependences • Many bha files have been deleted • Can only delete files - not DB2 records • Suggested procedure • Reprocess as needed • Handle completeness of reprocessed data set as in case of 2004-2005 data (except for bgg files) • Delete files (not DB2 records) for previous DBVs • Make sure older DSTs were based on complete reconstruction, otherwise delete (should not happen)

7. Other maintenance • Several runs reconstructed with incorrect DBV in DB2: • 68 runs reconstructed with DBV-21 recorded as DBV-20 • 4 runs reconstructed with DBV-25 recorded as DBV-24 • All deleted - easier to reprocess than to update all DB2 tables • Delete off-peak runs reconstructed with DBV-24 • Track mass evaluation in EvCl assumes √s = 1019.4 MeV • All scan data, small part of 1000 MeV data • 342 runs total

8. Reconstructed KLOE data

9. Monte Carlo tests • First production test 2 Nov: • 25 pb–1 all_phys, LSF = 0.2 (Runs 37000-37200) • VERT banks corrupted, 1/4 endcaps missing! • Second production test 25 Nov: • Problems with VERT banks, endcaps fixed • Same sample as first test - not enough events • Timing problem for EmC background • Third (“preproduction”) test 20 Feb - 13 Mar: • 100 pb–1 all_phys, LSF = 0.2 (Runs 39000-39600) • Test new cluster efficiency parameters • Test new EmC time resolution parameters • Additional EmC studies on energy scale calibration • Other additions and modifications added to test

10. Items we feel good about

11. Confirm/check status of these items

12. Monte Carlo production plans Averaged over entire MC sample: 0.21M evts/B80 day = 2.4 Hz 0.41 s/evt (simulation + reconstruction + DST) Estimated time for 2004-2005 MC 2001-2002 MC production f  all, scale = 0.2 KSKL, scale = 1 K+K-, scale = 1 f radiative, scale = 5 Other (1M evts/pb-1) Total: 3.1M evts/pb-1 (about same as number of f decays in data)

13. Offline resources: CPU • Notes: • 1 “B80” CPU = 1 Power3 375 MHz installed in B80 server • 1 P4+ CPU = 2.4 B80 CPU (scaling GEANFI execution time) • Fairly well-known estimate • Reconstruction runs even faster than GEANFI on P4+ iff DH smooth • 1 P5 “CPU” = 0.86 P4+ CPU (GEANFI, 100 pb-1 production test) • Less well known: many issues during production test • DH in bad shape, other processes competing for P4+ time? • Use old P4+ numbers, results consistent with PS & FF 1-Feb-06

14. Offline CPU needs Up to ~300 B80 available for offline, leaving 80 to users 283 days of offline work iff CPU (and not DH) limiting factor

15. Impact of DH problems • From a timing analysis of past production and 100 pb-1 test: • tCPU(MC) = 210 B80 ms, tCPU(rec) = 180 B80 ms • tDH = negligible in past, tDH = 90 ms now • Assumes tDH doesn’t get any worse when we use more CPUs! • 54 slots ~100 B80 for 100 pb-1 test + online reconstruction running • Maintaining 80 B80 for users leaves 200 B80 for offline • 425 days (instead of 283 days)

16. Various issues (HepDB & DB2) • Missing MC entries in C1 calibration database (HepDB) • DC - all calibration banks • EMC - CCAV, QCAV banks • Patch applied in RTPATH: For MC, always access CN database • HepDB to DB2 migration! (FS) • Are all calibrations in their final state? • Need to copy dead/hot wire maps and efficiencies from test table to final table • Need to get all bgg and lsb files into disk cache • Generally, review selection of file types maintained on disk

17. Questions What work needs to be completed before MC production starts? Do we need any further MC production testing? How many CPUs do we use for MC/reprocessing What data do we reprocess first? What can we do to improve DH system performance? Do we start the HepDB to DB2 migration now?