1 / 13

CMS Status and Plans

CMS Status and Plans. Progress towards GridPP milestones Workload management (Imperial) Monitoring (Brunel) Data management (Bristol) Future plans CMS Data challenge Issues Network performance EDG. Bristol, Brunel and Imperial (1.5 GRIDPP FTE in total). CMS deliverables 2002-3.

enye
Download Presentation

CMS Status and Plans

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMS Status and Plans • Progress towards GridPP milestones • Workload management (Imperial) • Monitoring (Brunel) • Data management (Bristol) • Future plans • CMS Data challenge • Issues • Network performance • EDG Bristol, Brunel and Imperial (1.5 GRIDPP FTE in total) CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  2. CMS deliverables 2002-3 • Table shows the deliverables and milestones and our progress in the first year of the GRIDPP project. CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  3. Production Monte Carlo GRIDPP deliverable 3.4.1 “Production Monte Carlo for CMS” achieved on 30/9/2002 • Key work since then was the “Stress Test” • Verification of the portability of CMS production environment into a grid environment; • Verification of the robustness of the European DataGrid middleware in a production environment; • Production of data for physics studies of CMS, possibly at the level of 1 million simulated events, which corresponds to (assuming 250 events per job): • 8000 jobs (4000 CMKIN and 4000 CMSIM) submitted to the system; • 8000 files created and registered to the Replica Management System (4000 ntuples and 4000 FZ files); • about 11 CPU years on a PIII 1GHz CPU • about 1.8 TB of data in FZ files CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  4. Production Monte Carlo CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  5. Production Monte Carlo Imperial, UK CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  6. Prototype Tier 1 centre • GRIDPP deliverable 3.4.2 “Prototype Regional Data Centre” is currently late with an estimated delivery date of 28/2/2003 • Large scale testing of grid data management software took place as part of CMS Stress Test • first time all components necessary for the Regional Data Centre available • 2,147 fz files actually produced: ~ 500Gb data successfully transferred using automated grid tools during production, including transfer to and from mass storage systems at CERN and Lyon • Replica Catalog (RC) service failed completely when the number of files registered was still significantly below that necessary for a Regional Data Centre • even when functioning, slow performance of RC was major bottleneck on file transfer speeds (eg. for a 200Mb file, could take 2mins to transfer, of which only 20s was actually file copying) • EDG have already planned replacement for the RC. This should fix problems, but is not scheduled for release until Testbed 2 CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  7. Adding RGMA to BOSS GRIDPP deliverable 3.4.3 “Initial Monitoring” achieved on 29/11/2003 • BOSS is the job submission and tracking system used by CMS but the monitoring part of BOSS is not “GRID enabled” • Delivered Functionality • A dummy BOSS producer was written and used to check the transfer of job monitoring data via R-GMA from Imperial to Brunel (and vice versa) and for tasks lasting >24 hours. • An R-GMA producer has been integrated into the BOSS job wrapper dbUpdator, and a separate “receiver” written that consumes information from R-GMA and updates the local BOSS database. • The producer/consumer have been tested successfully with multiple simultaneous jobs running on a machine at Brunel. Deployment on to the testbed was postponed until after the CMS stress test and is now in progress. CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  8. Adding RGMA to BOSS Only the interface classes are shown, much more information is available from our web pages at http://www.brunel.ac.uk/~eestprh/GRIDPP/Index.htm CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  9. Grid Object Access • Looking for a replacement for Objectivity. • Work commenced on the POOL project. • First internal release Oct 2002. • First public release Dec 2002. • Bristol is developing applications to stress-test POOL public release as a CMS project. • Excellent crossover with BaBar as Bristol is also involved with redesign of the BaBar event store. • BaBar is evaluating POOL as possible candidate framework. CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  10. Future: Data Challenge 2003-4 • DC04 is a crucial milestone for CMS computing • An end-to-end test of our offline computing system at 25% scale • Simulates 25Hz data recording @ 2.1033 luminosity, for one month • Tests software, hardware, networks, organisation • The first step in the real scale-up to the exploitation phase • Data will be directly used in preparation of Physics TDR • Grid middleware is an important part of the computing system. What is used in the next few months is likely to be what will be used until 2006-7 (no refactoring). • DC03 in the UK (starts July 03, five months) • Plan to produce ~50TB of GEANT4 data at T1 and T2 sites, starting July ’03 • All data stored at RAL: this means 60Mb/s continuously into the RAL datastore for 4-5 months • Data digitised at RAL with full background; 30TB of digis shipped to CERN at 1TB/day (>100Mb/s continuously over WAN) CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  11. Data Challenge 2003-4 • Some very serious technical challenges here • The work starts now; CMS milestones oriented accordingly • If Grid tools are to be fully used, external projects must deliver CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  12. Network Performance • Networks are a big issue • All Grid computing is reliant upon high-performance networks • But: data transfer was a bottleneck in previous data challenges • Some initial progress in this area in 2002 • CMS (and BaBar) using smart(er) transfer tools with good results • Contacts made with PPNCG / WP7 and UCL • Starting to get a feel for where the bottlenecks are • Most often in the local infrastructure • Progress in real understanding is being made CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

  13. Summary • Sucesses • RGMA Producer and Consumer tools for BOSS job monitoring were released by Brunel.  • ORCA-7, built upon a prototype of the new root-based persistency solution, has been released, and initial performance testing of this release is in progress at Bristol. • New Grid-enabled Impala & Boss framework released, used during CMS stress test at RAL and Imperial. • Some “CMS” GRIDPP software being evaluated by BaBar UK • Problems • Deliverable 3.4.2 has slipped, and is now expected to be achieved by the end of Feb ’03. The primary reasons for this slippage are: • very late release of “stable” EDG software • functional problems with the data management components of the EDG release. These risks were noted in the original deliverables planning. CMS Report – GridPP Collaboration Meeting VI Peter Hobson, Brunel University 30/1/2003

More Related