1 / 31

Grid Event Management Using R-GMA Monitoring Framework

Grid Event Management Using R-GMA Monitoring Framework. ISGC April 27-29, 2005 Min Tsai ASCC/CERN. Overview. Introduction to R-GMA R-GMA Monitoring Framework Event Management System. R-GMA Introduction (Slides from Steve Fisher). Background R-GMA developed in EDG Deployed by LCG

zaza
Download Presentation

Grid Event Management Using R-GMA Monitoring Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid Event Management Using R-GMA Monitoring Framework ISGC April 27-29, 2005 Min Tsai ASCC/CERN

  2. Overview • Introduction to R-GMA • R-GMA Monitoring Framework • Event Management System

  3. R-GMA Introduction (Slides from Steve Fisher) • Background • R-GMA developed in EDG • Deployed by LCG • Re-engineered for the EGEE project • Uniform data transport mechanism for: • Measurement • Resource information • Logging • Monitoring

  4. R-GMA – one cloud • A relational implementation of GMA (from GGF) • Powerful data model and query language • All data modelled as tables • SQL can express complex queries in one expression • Creates impression that you have one RDBMS Producer R-GMA Consumer

  5. Site A Site B Server Server Registry Service Schema Service Registry Service Key control data Example Deployment Server Server Producer Service Consumer Service Producer Service Consumer Service API API Consumer application Producer application Site C Site D

  6. Producer application API Key control data The Producer Service • Producer created declare Producer Service

  7. Producer application API Key control data The Producer Service • Registry Service contacted to register producer Registry Service register producer declare Producer Service

  8. Producer application API Key control data The Producer Service • Data transferred from the application to Producer Service • If there are no consumers then the data goes no further than the Producer Service Registry Service register producer declare Producer Service insert

  9. Primary Producers (PP) Initial source of data Data published by user code stored by Producer Service Secondary Producers (SP) Used to republish data to: co-locate information to speed up queries reduce network traffic SP PP PP Producers PP

  10. Consumer application API Key control data The Consumer Service • Query issued by user code • Consumer Service carries out all of the work on its behalf query Consumer Service

  11. Registry Service Mediator Consumer application API Key control data The Consumer Service • Registry Service contacted to identify relevant producers • Mediator works out list of producers to answer query list of producers query query Consumer Service

  12. Registry Service Mediator Consumer application API Key control data The Consumer Service • Consumer Service contacts relevant producers list of producers query query query Producer Service Consumer Service Producer Service Producer Service Producer Service

  13. Registry Service Mediator Consumer application API Key control data The Consumer Service • Data transferred directly from Producer Services to the Consumer Service list of producers query query query Producer Service Consumer Service Producer Service Producer Service tuples Producer Service tuples

  14. The Consumer Service • Query types: • Continuous queries • as soon as new data becomes available it is broadcast to all interested parties • History Queries • return time sequenced data • Latest Queries • correspond to intuitive idea of current information

  15. Using R-GMA • Three steps to Producing • Create Producer • Declare table • Insert data • Three Steps to Consuming • Create a Consumer with your query • Start it • Read data • More information • JRA1-UK web site: http://hepunx.rl.ac.uk/egee/jra1-uk/

  16. R-GMA Monitoring Framework

  17. Problems with Monitoring Tools • Inconsistent site configuration sources • Results in different site coverage • Difficult to correlate test results • Different site Identification methods • Search through many web pages • Time consuming especially with 100’s sites!

  18. Unified Monitoring • Site configurations are consistent • Only from GOCDB • Data is sent to single data transport (R-GMA) • Shared data schema • Shared client applications (UI, Alarm Sys.) • Data can be selectively collected to create custom view (CIC, ROC, site admin) • Application can share results and build dependencies • Application can more easily be distributed using R-GMA to aggregate the data

  19. Unified Schema • TestDef • Test are registered here • Provides information describing each test • testName, friendlyName, isVirtual • dataType, Unit • testHelp, testTitle • TestDefRelation • Defines logical test groups and organize tests • superTestName, subTestName • displayPriority, flavour

  20. Unified Schema (cont.) • TestData • Holds data produced by test • testName • nodeName • summaryData • detailedData • Status • 0:NA, 10:OK, 20:INFO, etc. • TestDataRelation • Defines test data dependencies • superTestName, superNodeName • subTestName, subNodeName

  21. Monitoring Console

  22. Event Management System (EMS)

  23. Introduction to EMS • Goal of Operation Centers to maximize availability of Grid services • EMS will help facilitate this by providing: • immediate notification as faults and performance degradation is identified • Shared EMS services that enables • Correlation and analysis of historical events of disparate monitoring tools • Reduce redundant development of notification modules for Grid monitoring applications

  24. EMS Prototype • Web Services event transport and command client developed as a proof of concept • Individual monitoring application send events directly to a centralized EMS Core Service. • Client applications can then query the EMS Core for new events • Events are stored in an XML format and filtered using XPath expressions

  25. Migration to R-GMA Transport • Already fully deployed in LCG2 • R-GMA benefits • No additional work required by applications using R-GMA Monitoring Framework • Built-in data archiving • Automatic timestamps • Flexible SQL queries • Planned features • Security • Fully redundant directory and schema service

  26. Event Management System

  27. EMS Java Client

  28. Additional Details and Help Information

  29. Sorting by Columns

  30. What Lies Ahead • Not ready for operations use yet • Display • Filtering mechanism to customize event received • Display list of current active alarms • Event reduction • Based on test dependencies • Damping of flapping events • Notification • Email • Audio • Communication • Broadcast comments for a particular event

  31. Questions? • R-GMA Monitoring Framework • http://goc.grid.sinica.edu.tw/gocwiki/RgmaUnifiedMonitoringSystem • Piotr Nyczyk • Judit Novak • Andrey Kiryanov • Dave Kant • EMS • http://goc.grid.sinica.edu.tw/gocwiki/EventManagementSystem • Antun Gusev • Doug Chen • Mark Ho • Jeng-Hsueh Wu

More Related