1 / 15

PanDA Monitor Development

PanDA Monitor Development. ATLAS S&C Workshop b y V.Fine (BNL). ATLAS User driven development. Maintenance: I mplementing the new “Feature Requests” Fixing the “known” bugs Development:

eudora
Download Presentation

PanDA Monitor Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PanDA Monitor Development ATLAS S&C Workshop by V.Fine (BNL)

  2. ATLAS User driven development • Maintenance: • Implementing the new “Feature Requests” • Fixing the “known” bugs Development: • high priority, short term developments to address high priority needs of operations and analysis users • plotting system: “flot”-based plots (after investigating options) to add requested plotting features • ”enhanced” job info and statistics pages • jQueryintegrated and used as standard basis for implementing 'client intelligence' and highquality UI • ”json” introduced as standard basis for server->client data export • modular, OO design used in new developments for greater maintainability, reliability, flexibility, and easier reuse PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  3. ADC Monitoring Feb 16, 2011: http://indico.cern.ch/conferenceDisplay.py?confId=127099 “User request driven” approach DONE DONE DONE DONE DONE DONE DONE DONE

  4. Monitor: Before and After Panda Monitor Web Server URL QUERY DB SQL QUERY HTML Page Generators Panda Monitor Web Server URL QUERY DB SQL QUERY flot histograms json html/json Where: JQuery “generic” code Page Generators PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  5. Jaroslava’s feedback https://indico.cern.ch/getFile.py/access?contribId=2&resId=0&materialId=slides&confId=130719 Slide 5 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  6. Related Savannah tickets PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  7. Related Savannah tickets There were 72 Savannah tickets and a bunch of the E-mail communications Trend: the number of “Bug Report” declined , the number of the “Feature request” grew and right now it is on the level 2.5 request per week. There was no complain about quality /performance of the “plotting” approach and its implementation. The requests are coming from users, managers, shifters, PanDA developers Among them: • Reprocessing jobs’ time distribution • Analysis “merging” jobs monitoring • Flexible statistic against of any imaginable filtering per user / date / site / cloud region/ processingType … in the other words against of any job parameter the Panda DB records. There was no new “plotting”-related request recently. There is the growing number of requests for the new “Monitoring” tables (requests (#79437 - “High priority tasks” , #79618 - “Page with list of multicloud sites”) . We are taking in account this trend to set our development priorities. DONE DONE DONE PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  8. Site performance issue analysis https://savannah.cern.ch/bugs/index.php?80368 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  9. reco performance: Intel vs. AMD https://indico.cern.ch/getFile.py/access?contribId=10&sessionId=1&resId=0&materialId=slides&confId=119169 From: AlexandreVaniachine <Sacha.Vaniachine@cern.ch> Date: March 18, 2011 8:21:03 PM GMT+03:00 To: davidrousseau <rousseau@lal.in2p3.fr> Cc: Wensheng Deng <wdeng@bnl.gov> Subject: reco performance: Intel vs. AMD ….we observed a difference in HI reconstruction performance on Intel vs. AMD that is larger than the difference in the GHz.Do you think it would be interesting to investigate if there is such a difference on the level of particular algorithms? PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  10. DOE/NSF Bi-weekly Operations Meeting PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  11. ATLAS User Analysis in the US Cloud http://indico.cern.ch/getFile.py/access?contribId=6&resId=0&materialId=slides&confId=119945 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  12. Panda Monitor / Dashboard Panda Monitor / Dashboard cross references PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  13. Output Merging for Analysis Jobs https://indico.cern.ch/getFile.py/access?contribId=121&sessionId=13&resId=0&materialId=slides&confId=119169 PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  14. Summary After a month of the development followed by one monthof use Monitor is free of the “known” bug and has been enhanced with about 2 dozens of the new features requested by ATLAS community via Savannah system. It includes the graphical representation of the user selected information. • Supporting large Monitor user communities in operations and analysis • Quick response to up to 4-6 requests/week from ops and analysis users • Incrementally upgrading monitor design/function in the context of the production monitor Near term developments to come: • more work to identify common effort and common code with other monitoring activities • improve flexibility of data summary histograms • client side intelligence, ajax to improve response time and minimize DB load • server side modular python framework for easier support and integration with other monitoring PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

  15. Questions? PanDA Monitor Development. Software & Computing Workshop, Apr 4-9, 2011

More Related