1 / 15

Feasibility study – server side

Feasibility study – server side. Fernando H. Barreiro Megino Mattia Cinquilli Daniele Spiga Daniel C. van der Ster CERN IT-ES-VOS. News. Meetings with the experts to review the analysis frameworks used by ATLAS and CMS This week focusing on server side (PanDA server and CMS WMS)

tabib
Download Presentation

Feasibility study – server side

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Feasibility study – server side Fernando H. Barreiro Megino Mattia Cinquilli Daniele Spiga Daniel C. van der Ster CERN IT-ES-VOS

  2. News • Meetings with the experts to review the analysis frameworks used by ATLAS and CMS • This week focusing on server side (PanDA server and CMS WMS) • Thanks to Paul Nilsson, Tadashi Maeno, Steve Foulkes, Simone Campana & Eric Vaandering for their time • Information is tracked on our document • Now readable by anyone who has the link • https://docs.google.com/document/d/1PJDBuH4gd5w5CzUJ5n2i7uOaFrhcYo5-5YErQdpzjvo/ed • This presentation should give an overview of our findings and the recommendations so far • Please interrupt for discussion, questions and corrections

  3. PanDA architecture

  4. CMS analysis framework DISTRIBUTED CENTRAL

  5. Resource management and brokerage

  6. PD2P and rebrokerage

  7. Priorities and fairshares

  8. Data handling in the server

  9. Site status

  10. Bookkeeping and redundancy

  11. Ideas about Common Approach:Data I/O Issues • After comparing the functionalities, we asked each of the server experts how the systems could be used as a common solution • The existing tight coupling between the data management and WM systems was previously seen as a potential showstopper, so we focused on this

  12. PanDA Data I/O Solution • Could PanDA server handle CMS data? • Not a very tight DQ2/Panda coupling • New libraries would have to be written • Input files • Store just LFN in PandaDB • CMSSW queries the Trivial File Catalog (TFC) and stages the data • Panda pilot would use a no-op mover (CMSSW reads the LFNs directly) • Output files • Wrapper/pilot copies files to the SE • Place files according to Trivial File Catalog • Write LFN’s and storage site name back to PanDA • Still need optional DBS registration and external asynchronous stage out service

  13. CMS WMSystem Data I/O Solution • Could CMS WMSystem handle ATLAS data? • Data discovery/registration: • Not a very tight DM/WM coupling • Interface to DM service is pluggable • Input and output would remain responsibility of the pilot “wrapper script”

  14. Priority & Fairshare Issues • PanDA has flexible priority mechanisms which are implemented and used in production for a few years • CMS WMSystem priorities extend to the requests – nothing in the model prevents priorities from being implemented down to Local Agents

  15. Conclusions • This week was focused on PanDA and the CMS WMSystem at the server side • Main differences between the two systems • Complexity of the systems and levels of queuing • CMS designed a distributed architecture to achieve scalability and fault tolerance • PanDA has a simple, central architecture and it has demonstrated scalability • Clear tradeoffs: • Central service has global view/control but single point of failure • Distributed service has higher scalability reliability but lacks global view/control • Resource allocation • Dynamic brokerage in PanDA, more fixed in WMSystem given distributed character • No show stoppers detected and positive attitude seen • Next weeks: investigate pilot frameworks and glideInWMS

More Related