1 / 10

M2OProxy Details

M2OProxy Details. Andy Salnikov 2009-11-03. Monitoring What is monitored. Requests client ID (host IP address/port) request time request size database ID (name) Replies response time response size Monitoring record = Request + Response info record size: 28 bytes.

cytheria-zenon
Download Presentation

M2OProxy Details

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. M2OProxy Details Andy Salnikov 2009-11-03

  2. MonitoringWhat is monitored • Requests • client ID (host IP address/port) • request time • request size • database ID (name) • Replies • response time • response size • Monitoring record = Request + Response info • record size: 28 bytes Monitoring in M2OProxy

  3. MonitoringCollecting Information • In-memory ring buffer of monitoring records with pre-defined size • 100,000 entries by default • less than 3 MB • can be increased arbitrary • Depending on request frequency can hold several days worth of history • Very low performance impact • everything is done in memory • just one record per request/reply Monitoring in M2OProxy

  4. MonitoringReporting tools • Special SQL syntax handled by proxy itself: SHOW M2OMONITOR [LAST <INTERVAL>] [GROUP BY <INTERVAL>] [GROUP BY DbName] [GROUP BY ClientId] [GROUP BY ClientHost] [GROUP BY Opcode] • Result is returned as a regular SQL table • Can use any MySQL tool to query monitoring data • Simple but sufficient for many tasks Monitoring in M2OProxy

  5. MonitoringExample • Simple python script to display summary data • e.g. number of requests grouped by minute and database name: ------------------------------------------------------------------------------------------------------------------------------------------ Time ATLASDD ATLAS_CONF_TRIGGER_V2 COOL_CALO COOL_CSC COOL_GLOBAL COOL_INDET COOL_LAR COOL_PIXEL COOL_TILE COOL_TRT UNKNOWN ------------------------------------------------------------------------------------------------------------------------------------------ 2008-10-10 23:44:00 2179 1340 109 46 23 46 60 37 183 60 12 2008-10-10 23:45:00 398 843 0 0 0 0 0 0 0 0 0 2008-10-10 23:46:00 0 1498 0 0 0 0 0 0 0 0 4 2008-10-10 23:47:00 2551 838 177 46 23 68 80 37 295 72 8 2008-10-10 23:48:00 26 5 92 30 21 8 30 3506 43 62 0 2008-10-10 23:49:00 0 0 180 30 21 30 50 2744 155 52 0 2008-10-10 23:50:00 0 0 0 0 0 0 0 762 0 22 0 2008-10-11 00:14:00 0 0 62 12 8 12 20 2961 72 54 0 2008-10-11 00:15:00 0 0 0 0 0 0 0 4019 0 14 0 2008-10-11 16:27:00 0 2838 0 0 0 0 0 0 0 0 8 2008-10-11 16:28:00 0 0 45 46 46 92 120 74 331 120 8 2008-10-11 16:29:00 5128 1686 203 46 0 0 0 0 35 0 8 2008-10-11 16:30:00 26 0 0 0 0 0 0 0 0 0 0 2008-10-11 16:34:00 0 0 310 60 42 60 100 2847 310 148 0 2008-10-11 16:35:00 0 0 0 0 0 0 0 4165 0 0 0 2008-10-11 17:44:00 2481 0 139 0 23 55 71 55 183 60 8 2008-10-11 17:45:00 96 0 0 0 0 0 0 0 0 0 0 2008-10-11 17:46:00 0 0 70 0 0 20 60 328 60 22 0 2008-10-11 17:47:00 0 0 30 0 11 20 0 6054 85 42 0 2008-10-11 17:48:00 0 0 80 0 10 0 0 630 10 10 0 2008-10-12 17:34:00 0 12 0 0 0 0 0 0 0 0 4 2008-10-12 17:35:00 0 2826 0 0 0 0 0 0 0 0 4 2008-10-12 17:36:00 3228 0 248 92 46 92 120 74 366 120 16 2008-10-12 17:37:00 1926 1686 0 0 0 0 0 0 0 0 0 2008-10-12 18:33:00 0 0 310 60 42 60 100 6351 310 148 0 2008-10-12 18:34:00 0 0 0 0 0 0 0 661 0 0 0 Monitoring in M2OProxy

  6. MonitoringSummary • Very useful to be able to run existing tools to extract monitoring info • Should be able to run “arbitrary” queries • group by time, database, client, opcode, etc. • time intervals – last day, hour, etc. • results appear as pseudo-table • Higher level info, such as database name, is extremely useful Monitoring in M2OProxy

  7. Client Client Client Client Thread Client Thread Client Thread ORACLE ORACLE ORACLE Threading model – M2OProxy Main Thread accept() • Thread-per-connection model • Main thread loops on accept() • Spawns new thread for every new connection • New thread is serving all requests from that connection • Every thread has separate ORACLE connection Monitoring in M2OProxy

  8. M2O threading model • Threads do not normally communicate • Very little synchronization needed • Schema cache • Monitoring stuff • No concurrency in the same connection, all packets processed sequentially (MySQL feature) Monitoring in M2OProxy

  9. CoralServerProxy threading Main Thread accept() • One client reader thread per connection • Single client writer thread • Single dispatcher thread • Single Server writer thread • Single server reader thread Server Writer Client Reader Client CoralServer Dispatcher CacheMgr Client Client Client Writer Server Reader Monitoring in M2OProxy

  10. CoralServerProxy threading • Much more concurrency in the system • even per-connection concurrency • More synchronization points • Packet cache • Connection manager • Packet queues • Potentially more problems • connections initialization and tear-down • not yet working as planned Monitoring in M2OProxy

More Related