1 / 1

ProdAgent

UCSD. Caltech. Open Science Grid (OSG) Mission. CMS Operations on OSG. OSG Usage : CMS vs Others. OSG is a US based distributed computing grid for data-intensive research.

keefe
Download Presentation

ProdAgent

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. UCSD Caltech Open Science Grid (OSG) Mission CMS Operations on OSG OSG Usage : CMS vs Others • OSG is a US based distributed computing grid for data-intensive research. • OSG goal is to promote collaboration in data-intensive research by providing a computing facility and services that integrate distributed, reliable and shared resources to support computation at all scales. • The computing and storage resources from campuses and research communities into a common, shared grid infrastructure via a common set of middleware. • Resources are owned and managed by the contributing members of OSG. • Among others, CMS T2 (and T3) sites in the US are members of the OSG. CMS (T1 + T2 + T3) CMS Usage of the Open Science Grid and the US Tier-2 CentersAjit Mohapatra, University of Wisconsin, Madison(On Behalf of CMS Offline and Computing Projects) CMS MC Production in OSG Routing CMS Jobs to OSG Running/Idle Job Monitoring • Centralised installation/maintainance of CMS Software package (CMSSW) at the OSG sites that participate in MC production. • MC production is handled by CMS specific “ProdAgent” software and managed by several teams based at CERN and Wisconsin in LCG and OSG regions, respectively. • Overall management of MC production and data transfer are performed by the CMS DataOps group. • Production in OSG (including 7 CMS T2s in the US) are handled by ProdAgent servers based at Wisconsin. • Utilization of various monitoring/debugging tools alongwith timely/effective communication with site admins has enabled us to run production efficiently. • ProdAgent creates and hands over jobs to Condor. Condor Schedd routing policy ProdAgent Job JobRouter for OSG site X max Jobs max Idle site Y max Jobs max Idle … Condor submit Job Routed to site X GRAM Site X Site Y Site Z Production Stats(LCG & OSG) Contribution of US (CMS) T2s Job Quality : US (CMS) T2s Jan’ 08 - Feb’ 09 Summer08/Fall08/Winter09 Summer08 / Fall08 / Winter09 & Part CSA07 / CSA08 T2_Region_FNAL (OSG) ~ 1B evts ~ 2B evts Jan’ 08 - Feb’ 09 CMS Analysis @ US T2s Analysis Jobs @ the T2s Summary & Plan • Terminated jobs stats according to CMS dashboard • Data hosted by the T2s • User Analysis : • CRAB(CMS Remote (Analysis Builder) Jobs • Local Analysis Jobs • Private (user) production • OSG provides packaged, tested and supported collections of software for the CEs and SEs for uniform operation across all the participating sites. • OSG has already been used successfully for CMS MC production for last 2 years. • Effective utilization of OSG (USCMS T2s) for MC production. All the US T2s have been very productive, stable and highly reliable. • While OSG itself provides supports to sites for grid related operations, effective communication between sites and CMS production operators are important to address production issues at sites in timely manner. • Expand the scope of MC production to CMS T3s (T3s are already used for user analysis) and other opportunistic resources which are also part of OSG. June’ 08 - Feb’ 09 Caltech (176816) UCSD-b (261475) Florida (159696) Wisc (434008) UCSD (312131) MIT (44353) • Effective and reliable usage of site resources depends strongly on the support and timely response at/from the site. Nebraska (65711) Purdue(198128) Computing in High Energy Physics, Prague (Czech Republic), March 2009

More Related