ADC monitoring tools

1. ADC monitoring tools Laura Sargsyan (ANSL (YerPhI (AM)) 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

2. If I have a grid site how can I monitor site activities? You can use ATLAS Distributed Computing monitoring tools! ADC monitoring tools can help sites from smallest to biggest 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

3. http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteview • monitor the overall status of the site • look into the site history • SSB aggregates information from different sources and provides • 240 different metrics • 18 production views • 26 devviews Specify a view Start with the SSB(Site Status Board) 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

4. http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteview#currentView=Cloud&highlight=falsehttp://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteview#currentView=Cloud&highlight=false Click on the sell to open page with detailed view Clock icon indicates that data is outdated • Click on the header • left click: history plots • right click: column info Site Status: cloud view 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

5. Monitor transfers, find failures • Go from global view to more detailed problem in NDGF-T1 Monitor data distribution status and progress with DDM dashboard(Distributed Data Management) http://dashb-atlas-data.cern.ch/dashboard/ddm2/ Click on the error number to open the error logs 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

6. http://dashb-atlas-data.cern.ch/dashboard/ddm2/#tab=transfer_plotshttp://dashb-atlas-data.cern.ch/dashboard/ddm2/#tab=transfer_plots filtering, grouping … possibilities DDM transfer plots 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

7. http://atladcops.cern.ch/drmon/ftmon_T1-T2_matrix_day.html Check how Functional tests (FT)are going on 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi • FT validate the site capability of successfully executing user and production jobs • FT results are collected and used to evaluate site performances ( problematic sites are excluded from the PanDA brokerage system)

8. Could be accessed from "SW_critical” view of SSB • http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteview#currentView=SW_critical&highlight=false Look onSoftware installationvalidationmonitor 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

9. http://bourricot.cern.ch/dq2/deletion/ DDM deletion service provideslive graphicalstatisticalreportsaboutdeletionprocess Deletion activity overview (NL cloud) Dataset, Gb to be deleted Activity at cloud Amount of deleted datasets per site, per disk type Amount of deleted datasets, files Check data deletion on your storage using DDM deletion services UI Number of errors 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

10. http://panda.cern.ch UsePanDA monitor(Production and Distributed Analysis) 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

11. http://panda.cern.ch/server/pandamon/query?dash=prod Production activity on the site AM-04-YERPHI for last 12 hours Check if jobs run correctly on PanDAmonitor Click to get more detailed view 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

12. httpd://dashb-atlas-task.cern.ch/templates/task-prod/ • Define time/time range • Select site • click to submit The result page: List of all tasks running on the selected site during defined time Dashboard Production monitor (task centric monitoring) Graphical representation of job statuses Click to open Summary plots

13. Monitor users tasks using Analysis Task Monitor https://dashb-atlas-task/task-analysis/ https://dashb-atlas-task.cern.ch/templates/task-analysis/#user=default&timerange=lastMonth&demo=on Task Monitoring collects and exposes a user-centric set of information to the user regarding submitted tasks (task concept based on output container) Click on the task name and open jobs list • New functionality added (testing phase) • killing on the task, site and individual job level • from the user web interface 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi 13

14. Check job accounting data using • Historical Views http://dashb-atlas-job.cern.ch/dashboard/request.py/dailysummary Check how much your jobs succeed or failed using ‘Success/Failure’ plotting Data is available in machine readable format via direct links Click on the header and choose a link 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

15. Check transfer errors details from DDM dashboard to find a reason of this failure Replica files were deleted after storage upgrade Check why jobs were failed(Historical Views web UI ) 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

16. http://dashb-atlas-sum.cern.ch/ Look SAM/Nagios test results on SUM(Site Usability Monitor) 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

17. http://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteview#currentView=Frontier_Squid&highlight=falsehttp://dashb-atlas-ssb.cern.ch/dashboard/request.py/siteview#currentView=Frontier_Squid&highlight=false Check Frontier/Squid using SSB 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

18. http://adc-monitoring.cern.ch ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

19. SquadHowToTWiki • https://twiki.cern.ch/twiki/bin/view/Atlas/SquadHowTo ADCoSTwiki • https://twiki.cern.ch/twiki/bin/viewauth/Atlas/ADCoS Documentation 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi

20. Acknowledgement Many thanks to D.Barberis, D. Benjamin, J. Schovancova, I. Ueda, for help, comments, suggestions during making this presentation. 2nd ATLAS-SouthCaucasus Software/Computing Workshop & Tutorial. Tbilisi