1 / 4

Network Statistics Optimization Ideas for Efficient Data Collection

Explore how to optimize network statistics collection, address scalability concerns, and improve data fetching practices for enhanced efficiency. Learn about adjusting periodic intervals, scheduling, and managing performance levels effectively.

fmcwhorter
Download Presentation

Network Statistics Optimization Ideas for Efficient Data Collection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistics Collection Optimization Ideas

  2. Existing Implementation: • Statistics Manager sends following statistics request per switch/node • Flow Statistics • Aggregate Flow Statistics • Table Statistics • Port Statistics • Queue Statistics • Group Statistics • Group Description • Meter Config Statistics • Meter Statistics • Currently Statistics Manager send these requests periodically with the time interval of 15 seconds for all connected nodes. • Statistics Manager cache few details about the nodes and flow tables per node, to avoid unnecessary read from MD-SAL • Statistics Manager listen for create/remove events for above mentioned switch entities from MD-SAL, and sends respective statistics request to fetch the latest statistics.

  3. Scalability Concerns: • How much data Statistics Manager fetches per node: • **Flow Statistics - [(56 * No of Flows per Table)* No of Tables] Bytes • ^^Aggregate Flow Statistics - [24 * No of Tables] Bytes • ^^Table Statistics - [24 * No of Tables] Bytes • ^^Port Statistics - [112 * No of Ports] Bytes • **Queue Statistics - [(40 * No of Queues per Port)* Number of Ports] Bytes • **Group Statistics - [56 * No of Groups] Bytes • **Group Description - [24 * No of Groups] Bytes • **Meter Statistics - [56 * No of Meters] Bytes • **Meter Config Statistics - [20 * No of Meter] Bytes • ** Statistics data will vary according to the number entries in their respective tables • ^^ Statistics data will be relatively constant per node • MD-SAL transactions per node - per statistics collection cycle : 9 • How many nodes are connected to the controller • More nodes, more data [e.g 100 nodes, 900 transaction/statistic collection cycle] • How frequently we fetch statistics data • Higher frequency, more data [Do the math from above example] • Pattern of how we send requests • Sending statistics request for all node in single shot will invite huge amount of statistics data in response.

  4. How to address these concerns: • It needs to be addressed at following levels • Better scheduling at nodes level • Don't send statistics request for all the nodes in one shot • Do we have any performance number about how many write transaction/second MD-SAL can handle? StatsManager should not be allowed to consume more then 30% of it, and accordingly we can determine how many switches we can send statistics request in one shot. • Adjustable periodic interval • Adjust interval time according to - { Number of connected nodes , Number of variable entities [Flow,Meter,Group,Queue]} • Different periodic interval for each statistics type • Flow statistics is crucial compared to aggregate flow stats • Keep the initial periodic interval High for low priority statistics • and make it adaptive according to number of entries • Welcome to new ideas :)

More Related