1 / 10

Site Status Board with Elastic search Ivan Dzhunov

Site Status Board with Elastic search Ivan Dzhunov. What we want to do and how we do it. Compare Oracle and Elastic search for storing and retrieving SSB metric data We need 2 dev SSB instances running on exactly the same data

giona
Download Presentation

Site Status Board with Elastic search Ivan Dzhunov

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Site Status Board with Elastic search Ivan Dzhunov

  2. What we want to do and how we do it • Compare Oracle and Elastic search for storing and retrieving SSB metric data • We need 2 dev SSB instances running on exactly the same data • Mechanism that synchronizes metric definitions and metric data on both instances Thanks Jacobo! • On the first (ES) instance • Use Elastic search for metric store • Use Oracle for everything else • topology info, metric/views definitions, admin rights, notifications, etc. • On the second (Oracle) instance • Use Oracle for everything – current production SSB Ivan Dzhunov

  3. Details for SSB ES implementation • New ES dataDAO class • responsible for writing and reading metric data to and from ES • using pyes library and Elasticsearch REST API • Configure (per SSB instance) which dataDAO to be used when writing and reading metric data Ivan Dzhunov

  4. What we already have in ES SSB • Last 1 month of SSB preproduction data (and growing) • Import of data • No overlaps • Sometimes tiny white gaps in historical data, due to not available ES server while making the inserts • Retrieve of data • Actions for metric history • default status and value plots • table with statistics • ranking plot • Action for expanded table • Action for index page • Actions for site history • default status and value plots • table with statistics Ivan Dzhunov

  5. Data retrieval speed comparison 1 Index page action - load times in seconds Ivan Dzhunov

  6. Data retrieval speed comparison 2 Expanded table action - load times in seconds Ivan Dzhunov

  7. Data retrieval speed comparison 3 Metric history action - load times in seconds Ivan Dzhunov

  8. Data retrieval speed comparison 4 Site history action - load times in seconds Ivan Dzhunov

  9. Summary of speed comparison • SSB current overview – index page and expanded table • Oracle is faster ~ by factor of 2 • still ES load times are very acceptable • Milliseconds for index page • Up to 1 second for expanded table • SSB historical data per metric or site • ES is much faster • Oracle queries may take minutes Ivan Dzhunov

  10. Still to be done for full ES SSB implementation • Fix the problem with stability of the cluster (sometimes the cluster is not available on inserts) • Also retry mechanism ? • Metric history : Number of sites plot - 1.5 day • Metric data modifications - 1 week • Alarms to sites on status changes - 1.5 day • Other areas where there is change of metric data in unusual way • Topology Maintenances collector – 1 day • Under Investigation functionality – 1 day • … Ivan Dzhunov

More Related