Status of Computing for the HI Program

1. Status of Computing for the HI Program RaphaëlGranier de Cassagnac LaboratoireLeprinceRinguet Ecole polytechnique – CNRS/IN2P3 for the CMS-HI Group Version V1, April 27 23:15Geneva time CMS Offline and Computing Workshop

2. Outline of Session • RaphaëlGranierde Cassagnac: General Status and Issues • Ivan Cali: DAQ Considerations for HI Running • Edward Wenger: HI Software Status CMS Offline and Computing Workshop

3. Overview of HI Computing: DAQ and T0 • Anticipated Start of HI Running • Change to Pb + Pb acceleration expected in mid-October • Two weeks of set-up time • Physics data taking expected to begin on November 1 • Pb + Pb running for 4 weeks, assuming 106 seconds live time • Special DAQ Considerations (Ivan’s talk) • Non-zero suppression of ECal and HCal, leads to xx MBytes/event • Zero suppression to be achieved off-line at the T0, leads to 3 [?] MB/event • Work flows and CPU requirements for this task are not yet determined • Data Processing • AlCa and DQM to be performed as in the pp running • DQM for HI being advanced by Julia Velkovska (at Vanderbilt) and Pelin Kurt (at CERN) • Prompt reco of the zero-suppressed files at the T0, ~7 days total • Leads to 1 MByte/event • ~400 TB of raw data and prompt reco files transferred to FNAL T1 tape archive • Processing after FNAL is still being considered (see next slide) CMS Offline and Computing Workshop

4. Overview of HI Computing: Post-T0 • Analysis and Re-Reco • Original plan in 2009 was to have a new, enhanced T2 center at Vanderbilt,and a satellite HI simulation center at MIT, plus 4 non-US T2 centers • Delays in DOE-NP funding have forces us into an alternate strategy • Original Plan • Presented to DOE-NP for review in May 2009 • Review reports released in September 2009 • Reports raised valid questions about the integration of CMS-HI computing with rest of CMS • A revised plan was requested to be submitted by the end of 2009; done in mid-January • A second, on-site review is expected at Vanderbilt in May but not yet final • We conclude that it is too late to count on operational hardware at Vanderbilt in November • Instead, the MIT center which did receive HI funds in 2010 will be used primarily • Modified Plan (only for the initial 2010 data processing) • MIT center will receive prompt reco files from FNAL, instead of Vanderbilt • Initial analysis and re-reco processing will be done at MIT • Other, non-US T2 sites will also be considered for analysis processing • Questions of connectivity either with MIT or with FNAL directly (?) CMS Offline and Computing Workshop

5. MIT Center Capabilities for HI Data Processing in 2010 - 2011 • CPU Power • Disk Space • Production Infrastructure • Personnel Commitments CMS Offline and Computing Workshop

6. Luminosity estimate for first beams:Shown by Bolek Wyslouch at MB Meeting, April 12 • AA-inelastic ~ 7.7b • Luminosity • L2010,peak(EPb/Z=3.5TeV) ~ 1 x1025 cm-2s-1 • <L2010>run ~ <L2010,peak • Avg. collision frequency 50-100 Hz: write all events to tape: Min Bias • Expect O(100M) events on tape [WHAT SIZE FOR EVENTS, from Ivan?] • Nominal luminosity (>2011): 8 kHz collisions: needs trigger Aperture limited CMS Offline and Computing Workshop

7. HI View of Personnel Responsibilities 1) It is important that all the CMS experts in each system shown on the left (Detector, Readout, ...) realize that their expertise will be critically needed during the HI run. For some systems, these experts will be almost completely in charge, while in others (RECO, SIM, ...) the HI persons will be taking the lead in giving directions. 2) In April we experienced the loss of a key HI person with knowledge of the T0 operations. It is very unlikely we can replace that person in time to make a difference in the summer preparations. So early, dedicated help for the T0 operations assisting the HI students and post-docs will be vital CMS Offline and Computing Workshop

8. Summary • To be written CMS Offline and Computing Workshop

9. [For Ivan:] Testing and Reviewing • The modification to CMS readout, DAQ, trigger and computing workflow will have to be tested • Different level of preparation/testing: • Simulation • Bench testing • Testing without beam • Testing with beam • Testing needs to be scheduled and included in the summer/fall activities, maybe earlier”: prioritization • HI group cannot do it alone but we are willing to put effort providing input for the relevant experts CMS Offline and Computing Workshop

10. [For Ivan:] Ideal test with pp beam • Time frame: June/July [already declined?] • With pp collisions • With PbPb trigger menu • SST readout in Virgin Raw or Processed Raw with zero-suppression in HLT • ECAL readout w/o zero-suppression to simulate PbPb conditions • Pixel readout with PbPb firmware • Transfer to SM at 100Hz or so CMS Offline and Computing Workshop

11. [For Ivan:] Summary of needs/wishes • Urgent work to get pixels and SST ready for HI beams • Conduct Heavy Ion Global Runs (HIGR) with and without pp beam, DQM in HI configuration • Modified reco workflow to include offline zero suppression • Good understanding of the temporary computing model and required resources CMS Offline and Computing Workshop

12. Backup Slides from Prior Meetings CMS Offline and Computing Workshop

13. Data Flow Model for CMS-HI: I • Principle Components of the Data Flow Model • In 2010 the change to Pb + Pb beams will start in mid-October • Heavy ion physics running will start in the beginning of November • Heavy run will last for 4 weeks • Anticipated live time of 7 x 105 seconds • Anticipated data rate of 70 Hz  ~5 x 107 events in minimum bias mode • Average minimum bias event size ~5 MB (50% physics uncertainty)after zero suppression (will take data initially unsuppressed, suppress at T0) • 250 TB of raw data expected (post-suppression, ~1 PB before suppression) • Prompt data reconstruction at the T0 • Data reconstruction times simulated to take a few days for the entire data set • Suppression time at T0 not known, but should be less than a few days total • Total T0 usage by HI group in November-December ~2 weeks during ~6 weeks • Prompt reconstruction output simulated as 150 TBytes • Net of 400 TB to be delivered from T0 to FNAL in ~6 weeks CMS Offline and Computing Workshop

14. Data Flow Model for CMS-HI: II • Principle Components of the Data Flow Model (2010) • Net of 400 TB to be delivered from T0 to FNAL in ~6 weeks • FNAL has agreed to be the second archive site for the HI data • Simplifies the workflow for data and prompt reco transfer from the T0 to the US • A new T2 site for CMS-HI has been proposed to the US DOE-NP • Site to be at Vanderbilt’s ACCRE supercomputer facility (2000 cores presently)This site is already a well-functioning T3 installation in CMS, running PhEdEx • The Vanderbilt T2 will subscribe the 400 TB of data on disk at FNAL, andthere already exists a 10 GB path from FNAL to Vanderbilt • The Vanderbilt T2 site will be sized to do both analysis and re-reco passes • Production files will be exported from Vanderbilt to selected T2 sites in CMS:MIT, France, Russia, Brazil, and Turkey who have active HI analysis groups • CMS-HI computing proposal was initially reviewed in May 2009 • Main comment was to have better integration with the rest of CMS computing,e.g. use of the FNAL T1 facility for tape archiving, use of T0 for prompt reco • Updated proposal submitted in mid-February 2010; now waiting for an on-sitereview to be scheduled by DOE at Vanderbilt, probably to occur in mid-May CMS Offline and Computing Workshop

15. Data Flow Model for CMS-HI: III • Principle Components of the Data Flow Model (2011) • A Pb+Pb run in 2011, after the pp run, is not yet announced • Reasonable to expect such a run, given that 2012 will have no run • Planning must be in place soon to have the resources to support such a run • Anticipated resources in 2010 (from data model document) • 7.7 x 1010 HS06-sec integrated CPU power distributed among Vanderbilt T2,MIT heavy ion data analysis center, and 4 selected T2 sites in CMS • 775 TB disk distributed among the same sites (450 TB at Vanderbilt) • 1.25 PB tape at T0, 0.70 PB tape at FNAL • Anticipated resources in 2011 (revised from data model document) • Shift run from 2012 into 2011, i.e. first use of the HLT for heavy ion running • 20 x 1010 HS06-sec integrated CPU power • 1.3 PB disk among all heavy ion centers • 2.0 PB tape at T0, 0.80 PB tape at FNAL CMS Offline and Computing Workshop