1 / 7

Reporting Metrics: Different Points of View (revisions and discussion)

This draft explores different points of view in reporting metrics for IP measurements, discussing the importance of understanding the audience and how the results will be used. It also covers long-term reporting, measurement aggregation approaches, and recommendations for reporting loss and delay metrics.

hildam
Download Presentation

Reporting Metrics: Different Points of View (revisions and discussion)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reporting Metrics: Different Points of View(revisions and discussion) Al Morton Gomathi Ramachandran Ganga Maguluri December3, 2007 draft-morton-ippm-reporting-metrics-03 “Our plans miscarry because they have no aim. When a man does not know what harbor he is making for, no wind is the right wind.” Seneca

  2. Different Points of View (POV): 2 key ones • When designing IP measurements and reporting results, MUST know the Audience to be relevant • Key question: “How will the results be used?” • Network • Characterization: • Monitoring (QA) • Trouble-shooting • Modeling • SLA (or verification) • Application Performance • Estimation: • Metrics Facilitate process • Transfer-dependent aspects • Modify App Design How can I _______ my network? What happened to my stream? DST SRC

  3. New Section: Long-Term Reporting • Section 6 now: • Summarizes results for each metric, loss, delay delay var. • Discusses Long-Term Reporting • Measurement Intervals need not be the same length as “Long” Reporting Intervals (days, weeks, months) • Long Measurements come with some risks • Temporary power failure: loose results to date. • Timing signal outage invalidating some measurements. • Maintenance on the meas. system, or its connectivity. • Relatively Short Meas. Intervals can help to • match user session length • allow dual-use of measurements in monitoring activities

  4. Approaches to Measurement Aggregation • Store all the singletons of the Measurement Intervals • Evaluate all singletons in the Reporting Interval • Methods like those envisioned in "Framework for Metric Composition", draft-ietf-ippm-framework-compagg-05, for Temporal Aggregation • Produce an estimate of the metric for the Reporting Interval using a deterministic process to combine the metrics from measurement intervals. • Use a numerical objective for the metric, and compare the results of each measurement interval: • Every measurement interval where the results meet the objective contribute to the fraction of time with performance as specified. • Present the results as "metric A was less than or equal to objective X during Y% of time.

  5. Ver 04 Resolved Steve Konish’s comments • Section 4 separates the discussion for delay for each point-of-view. Section 3 (Loss) would benefit from the same organization – synopsis at beginning and end. • New sub-sections to help with this. • What do you mean when saying the processing "forks" in this context: section 4.1.1 and again in section 4.3. • think “forks in the road” – no eating utensils in the text now • (4.3) ... where subsequent processing depends on whether the packet arrives or times-out,… • Other editorial comments • They all help, thanks!

  6. Summary of Recommendations so far: • Set a LONG Loss threshold • Distinguish between Long Finite Delay and Loss • Avoid truncated distributions • Delay of Lost Packets is UNDEFINED • Maintain orthogonality – avoid double-counting defects • Use conditional distributions and compute statistics • Report BOTH Loss and Delay • Report BOTH the Sample Mean and Median. • Comparison of the Mean and Median is informative • Means may be combined over time and space (when applicable) • Means come with a weighting function for each sample if needed, the sample Size, and Loss simply reduces the sample size • Means are more Robust to a single wonky measurement when the sample size is Large • Move the Industry Away from “Average Jitter” • Use the 99.9%-ile minus minimum PDV • Portray this as a Delay Variation “Pseudo-Range”

  7. What’s Next? • Homework for IETF-70 • Did you read either draft? • Proposal: • Make this draft the basis for non-short-term reporting • Complement to current (short-term) draft, without the restrictions brought-on by producing a result every 10 seconds

More Related