Metrics Framework draft-ietf-pmol-perf-metrics-framework-00.txt

1. Metrics Frameworkdraft-ietf-pmol-perf-metrics-framework-00.txt Author: Alan Clark alan @ telchemy.com IETF 72 - Dublin

2. Changes from 02 draft • Working Group draft • Incorporated comments on previous draft • Minor revisions throughout to get to a reasonable WG-00 draft IETF 72 - Dublin

3. Comments - Dan Romascanu • 1. Section 1.1: s/Although the IETF has two Working Groups/Although the IETF currently has two active Working Groups/ • Will change • 2. Section 3.3: s/Temporal Aggregation is a summarization of metrics into/Temporal Aggregation is a summarization of sets of metrics into/ • Will change • 3. Section 3.4.1 - it is not clear what is the difference between 'what the metric is' and 'describes the metrics'. Maybe the intent here is s/describes the metric/describes the rationale of the metric/? • Intent – “what” is a definition and “describes” is more explanatory • 4. (iii) Measurement Method - should not we relate here also to the measurement points? For example in the sip-performance-metrics draft some of the metrics are measured only at UACs, other at UAC or UAS, etc. • Yes – if the measurement method is specific to a measurement point IETF 72 - Dublin

4. Comments - Dan Romascanu • 5. The 2119 capitalization in section 3.4.2 is inconsistent. Maybe the MUST in the first phrase is enough and the rest needs not be capitalized. • Will capitalize SHOULD etc • 6. There are alignment problems with the paragraphs in 3.4.3 • Yes – will need Al’s help on the XML editor though!! • 7. 3.4.3 (ii) - is conformance testing really a SHOULD in our metrics documents? Or maybe just a MAY? I am hesitating on this, we define very little conformance testing in the IETF • Good discussion point for WG meeting • 8. 3.4.4 - if Conformance Testing is to stay, it should be part of the template. • Agreed • 9. It is not clear to me what is the scope of 3.5 and in any case I do not believe it should be located here in the document. Maybe its place is rather in section 4, as a checklist for reviewing PM documents, or maybe in an Annex. • Idea was to have a way to assess the quality of a metric definition • 10. The text in 3.6 and the whole subsection should be merged with the text about reporting models in 3.4.4 • Agreed IETF 72 - Dublin

5. Comments - Dan Romascanu • 11. Section 4.2 - What proposal approval are we talking about here? The approval of initial work, or the final approval? In any case I think that we just need to refer to the IETF processes to charter new work, or to approve documents. We do not intent to invent a new process here, but to either support other WGs with the expertise of a cross-area directorate, or create the framework of a new WG. I am not sure this section is needed at all. • Agreed – can just refer to existing approach • 12. Section 4.3 - here we should explain in a few words the two options 1. a cross-area directorate which is called to give advice on the development of new PMOL metrics in other WGs and reviews the documents as part of the WGLC and IESG review process 2. a dedicated WG within the OPS Area which creates the framework for other areas protocol experts to develop here their PM definition documents. In both cases there is a need to interact - in case #1 as a WG-directorate review team relationship, in case #2 as a protocol WG- PMOL WG relation. The decision should be left IMO to the IESG • Agreed • 13. Section 4.3 - I think that the requirement for a dedicated mailing list for each effort is exaggerated - the protocol WG mail list and/or the PMOL (directorate or WG) list should be enough. • Agreed – although could have an “if needed” IETF 72 - Dublin

6. Next Steps • Revise draft to incorporate Dan’s (and any other) feedback • Will get this submitted in 1-2 weeks • Comments? IETF 72 - Dublin