1 / 11

LcgCAF:CDF submission portal to LCG

LcgCAF:CDF submission portal to LCG. Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel Jeans, Subir Sarkar and Igor Sfiligoi. Head Node. CafExe MyJob.sh MyJob.exe. CDF Data Handling Model.

lamont
Download Presentation

LcgCAF:CDF submission portal to LCG

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LcgCAF:CDF submission portal to LCG Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel Jeans, Subir Sarkar and Igor Sfiligoi

  2. Head Node CafExe MyJob.sh MyJob.exe CDF Data Handling Model • Fermilab: host data and a farm used mainly to • analyze data, the CAF Central Analysis Facility • Remote sites: have a decentralized CAF (dCAF) where Monte Carlo are produced and later copied to FNAL, some sites (ex. CNAF) has data • CAF and dCAF are CDF dedicated farms structured: CAF User Desktop Monitor Worker nodes Submitter Developed by: H. Shih-Chieh, E. Lipeles, M. Neubauer, I. Sfiligoi, F. Wuerthwein Mailer CDF Italian Computing Group

  3. Moving to a GRID Computing Model • Reasons to move to a GRID computing model • Need to expand resources, luminosity expect to increase by factor 8 until CDF ends • Resource were in dedicated pools, limited expansion and need to be maintained by CDF persons • GRID resources can be fully exploited before LHC experiments will start analyze data • CDF currently access in Europe more than 5000 KSpecInt2000! No hope of having so many resources without GRID CDF Italian Computing Group

  4. User Desktop job job … output output LcgCAF: General Architecture GRID site CE and worker nodes CDF-UI CDF user must have access to CAF clients and Kerberos V ticket CDF-SE Robotic tape Storage CDF Italian Computing Group

  5. CDF-UI Job Submission and Execution WN Monitor … Job WMS WN Submitter Mailer • Monitor: provide job information to user • Submitter: create JDL. Retrieve user tarball and make it available to WN • Mailer: send mail to user when job is over CDF Italian Computing Group

  6. Interactive Job Monitor: CafMon kill CafMon tail CafMon dir CafMon log CafMon top CafMon ps Job Monitor CDF-UI Job information WMS Information System WN information WN WN: job wrapper collects information like load, dir, ps, top WMS : keeps track of job information CDF Italian Computing Group

  7. CDF Code Distribution Lcg site: CNAF Proxy Cache … … /home/cdfsoft /home/cdfsoft Lcg site: Lyon Proxy Cache • No CDF code or Run Condition Database need to be available on WN • On demand Parrot translates local warning in HTT requests towards Fermilab code server and DB • To be scalable when a site has many equal requests the information are cached near WN using squid CDF Italian Computing Group

  8. … … CDF Output storage gridftp Job output Worker Nodes CDF-SE KRB5 rcp CDF-Storage • From CDF-SE or CDF storage output copied using gridftp to FNAL fileservers. • An automatic procedure copies files from fileservers to tape and declare them to the catalogue CDF Italian Computing Group

  9. #segments succeeded # segments submitted e = Performances CDF B Monte Carlo production GRID failures: site miss-configuration, temporary authentication problems, WMS overload LcgCAF failures: Parrot/Squid cache stacked Retry mechanism: necessary to achieve ~100% efficiency WMS retry for GRID failures and LcgCAF “home-made” retry based on logs parsing CDF Italian Computing Group

  10. Future developments An interface between SAM (CDF catalogue) and GRID • services is under development. This will allow: • File declaration into the catalogue directly from LcgCAF • Automatic transfer of files from LCG sites to FNAL tape • Possibility of data analysis on LCG sites by single user CDF Italian Computing Group

  11. Conclusions • CDF has developed a portal to access the LCG • resources • Currently CDF access the following sites: • CDF has produced and is producing Monte Carlo data exploiting these resources demonstrating that GRID can serve also a running experiment CDF Italian Computing Group

More Related