1 / 23

WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre.

WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre.beche@cern.ch>. Outlines. Mission History of data transfer monitoring tools Data volume and retention policy Main improvements Future work Application usage. Mission.

lorene
Download Presentation

WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers. AlexandreBeche <Alexandre.beche@cern.ch>

  2. Outlines Mission History of data transfer monitoring tools Data volume and retention policy Main improvements Future work Application usage Alexandre Beche – GDB

  3. Mission ATLAS FTS + XRootD CMS FTS + XRootD LHCb FTS ALICE XRootD Alexandre Beche – GDB

  4. Mission ATLAS FTS + XRootD CMS FTS + XRootD LHCb FTS ALICE XRootD Alexandre Beche – GDB

  5. WLCG Transfers Dashboard User Interface Alexandre Beche – GDB

  6. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API / UI WLCG FTS Alexandre Beche – GDB

  7. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API / UI XRootD Integration WLCG FTS FAX AAA Alexandre Beche – GDB

  8. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API/UI WEB API/UI WEB API / UI XRootD Integration XRootD Detailed Dashboard WLCG AAA FAX FTS FAX AAA Alexandre Beche – GDB

  9. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API/UI WEB API/UI WEB API / UI XRootD Integration XRootD Detailed Dashboard WLCG AAA FAX EOS Integration FTS FAX EOS AAA EOS Alexandre Beche – GDB

  10. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API/UI WEB API/UI WEB API / UI XRootD Integration XRootD Detailed Dashboard WLCG AAA FAX EOS Integration ALICE Integration FTS FAX EOS AAA EOS ALICE Alexandre Beche – GDB

  11. WLCG Transfers DashboardFederated approach WEB API/UI WEB API WEB API/UI WEB API / UI ALICE FTS AAA FAX FTS FAX EOS AAA EOS ALICE Alexandre Beche – GDB

  12. WLCG Transfers DashboardFederated approach WLCG Transfers Dashboard API / UI WEB API/UI WEB API WEB API/UI WEB API / UI ALICE FTS AAA FAX FTS FAX EOS AAA EOS ALICE Alexandre Beche – GDB

  13. Data Volumesince June 2012 • Data are split into 3 schemas • FTS, XRootD (AAA and FAX) • High optimization (Partition, no Global indexes, …) Alexandre Beche – GDB

  14. Data Volumesince June 2012 • Data are split into 3 schemas • FTS, XRootD (AAA and FAX) • High optimization (Partition, no Global indexes, …) Alexandre Beche – GDB

  15. Data Volumesince June 2012 • Data are split into 3 schemas • FTS, XRootD (AAA and FAX) • High optimization (Partition, no Global indexes, …) • Snapshot on Tuesday 11 Feb 2014 • Shows the usage of the DB at this specific time • No conclusion can be drawn on technology usage • See next slide on data retention policy Alexandre Beche – GDB

  16. Data retention policy • Two kind of data: • Raw: • Kept for 3 months for FTS • Kept indefinitely for AAA and FAX • Statistics: • Different granularity (10 minutes, daily) • Less than 50GB of data • Kept indefinitely Alexandre Beche – GDB

  17. Main improvements • ALICE traffic is now included • No more duplication at the raw data level • WLCG view is an aggregation over APIs • FTS Dashboard is much faster • Does not have the XRootD overhead • Long time period can be queried • See next slide Alexandre Beche – GDB

  18. Activity since June 2012 Impossible with the old architecture Alexandre Beche

  19. Future work • Improve the integration of technology specific dashboards Alexandre Beche – GDB

  20. Future work • Integration of the map view • Already available for AAA, FAX and FTS Alexandre Beche – GDB

  21. Application usageincluding WLCG Home page Alexandre Beche – GDB

  22. Summary Thanks for your attention • Completely new architecture • Greater flexibility • Easier integration of heterogeneous data • Optimized technology-specific dashboards • Aggregated together inside a top level UI Alexandre Beche – GDB

  23. Useful links • AAA Dashboard • http://dashb-cms-xrootd-transfers.cern.ch • FAX Dashboard: • http://dashb-atlas-xrootd-transfers.cern.ch • FTS Dashboard: • http://dashb-fts-transfers.cern.ch • WLCG Transfers Dashboard: • http://dashb-wlcg-transfers.cern.ch Alexandre Beche – GDB

More Related