Schemas Update Network Measurements Working Group

1. Schemas UpdateNetwork Measurements Working Group Mark Leese VIA Richard Hughes-Jones GGF11, Hawaii, June 2004 Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

2. Contents • What is happening with the four sets of NM-WG schema work? • test/data request • data publication • service discovery • capability discovery Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

3. Request & Publication Schemas (1 of 500) test request (request schema) tests results (publication schema) • NM-WG have defined XML schemas for requesting network tests and historic data, and publishing network measurement data • Aims: to standardise communication, and… • …use XML, for web services and OGSI/WSRF model • Simple use case… • Request & result messages can be formatted using standardised schemas = truly powerful combination Network Monitoring Service Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

4. Request & Publication Schemas (2) • Current efforts are centered on: • Harmonising separate request and publication schema requirements... • http://forge.gridforum.org/projects/nm-wg/document/Measurement_Report_Schema_Requirements/en/ • http://forge.gridforum.org/projects/nm-wg/document/Network_Measurements_Request_Schema_Requirements/en/1 …which were previously independent documents. • Replace schema implementations in WXS (W3C XML Schema) language with new RelaxNG (from OASIS) versions: • much easier to read • better support for modularisation • Producing sample system implementations (albeit in WXS): • Warren Matthews for Internet2 (report at last GGF) • Paula Grosso for SLAC (report at last GGF) • Tanya Brethaur for NCSA • Producing “business logic” documentation: • overall aim is for a NM-WG compliant requestor to be able to drive predictable behaviour out of a compliant responder • requirements must directly/indirectly define a set of rules... "IF you receive X, THEN do Y" Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

5. Request & Publication Schemas (3) • Okay, flick between this slide and the next until you feel nauseas. This is in WXS language… <xs:element name="NetworkMeasurement"> <xs:annotation> <xs:documentation>Report of a network measurement.</xs:documentation> </xs:annotation> <xs:complexType> <xs:sequence> <xs:element name="networkCharacteristic" type="nm:NetworkCharacteristic"> <xs:annotation> <xs:documentation>The full name of the network characteristic being measured. </xs:documentation> </xs:annotation> </xs:element> <xs:element name="subject" type="nm:NetworkEntity"> <xs:annotation> <xs:documentation>The hosts and mnetworks that are being tested.</xs:documentation> </xs:annotation> …with pages and pages more to go! Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

6. Request & Publication Schemas (4) This is a complete top-level description in RelaxNG format… include "nm_subject.rnc" include "nm_result.rnc" <and a few other includes> start = element networkMeasurementReport { NetworkMeasurementReport } NetworkMeasurementReport = element networkMeasurement { NetworkMeasurement }+, attribute version { token }? NetworkMeasurement = element characteristic { token }, element subject { Node | Path }, element methodology { Methodology }, element results { ResultSet } ResultSet = element timeInterval { TimeInterval }, element result { Result }*, element statistic { Statistic }* Methodology = element endpoint { Endpoint }, element tool { Tool }, element parameterSet { ParameterSet }* Endpoint = token "source" | token "destination" • Hopefully you appreciate which is easier to read. Like regular expressions: ? is “zero or one” * is “zero or more” | is “or” Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

7. Request & Publication Schemas (5) Interesting points from recent discussions: • Request schema should now support requests for multiple characteristics in the same message. • Should be able to ask for statistical summaries, without getting all of the raw data as well. • Some would like the ability to say "Give me the last five raw values of this characteristic AND the average over the last day". Not possible given current request schema design. • Major debate (i.e. will hopefully to lead to good fist fight ;-) over batching of results. Example: there are 32 results for a particular request. "BatchSize" is 10. Results are returned as 3 batches of 10, and one batch of 2 results. Should this be stateful (both receiver and requestor must keep track) or not? • Serious discussion over what time format to use, mainly NTP vrs Unix timestamp. General consensus that we need to pick and stick with one method for now to avoid confusion. Unix timestamp has most software support, whereas NTP is a common device format. Does anyone in the room have comments on any of these? Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

8. Service Discovery • At least Brian Tierney working on this… • Where are the services that can run tests or provide measurement data? • Brian is more important than Mark, so “Service Discovery” has its own presentation ;-) Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

9. Capability Discovery (1) • Mark and Paul Mealor of UCL looking at this… • Once you have found a network monitoring service, what can it do for you? • Is iperf available? • Is iperf available with a 4MB buffer size? • Is iperf available at 10am on Monday (peak time)? • Does the service let you run tests at all? Is it just a source of historic measurement data? • Deducing what a service can do by sending requests and waiting for “yes” or “no” replies is not good enough. • We need dedicated way of asking. Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

10. Capability Discovery (2) range discrete • Current thinking is to use a modified request schema. • For example, you send… <capability request></capability request> <tool></tool> <toolVersion></toolVersion> <tcpBufferSize></tcpBufferSize> ... • …and get back... <tool>iperf, udpmon</tool> <toolVersion>6.1, 2.0</toolVersion> <tcpBufferSize>1048576-4194304</tcpBufferSize> ... • Mark and Paul will circulate ideas…please comment! Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

11. Capability Discovery (3) Issues: • Overlap with service discovery schema: when you receive a service discovery response it tells you a service(s) location and the metrics it can provide. Do we want to extend this to contain information on other capabilities? Brian has already said… “I was not sure if these [service discovery] schemas should include information on what probes are available...” (email, 21st April) • Assuming different users have different privileges (e.g. SysAdmin vrs grad student) capability reports need to be requestor specific, with all of the associated AAA issues. Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

12. Capability Discovery (4) • Do we even have the right requirements? • Being good computer scientists we’ve started out by considering what we want to do, regardless of how easy it is. That is, to provide the ability to ask what a system is prepared to do for a particular requestor at a particular time. • Do we want it this complicated? Do we really just want to know what values the receiving system will accept, signifying the system’s overall capability? • Should we have classed ideas from the Internet2 SMM NM-WG session (http://e2epi.internet2.edu/SMM04/ggf-nmwg.html) as out of scope? E.g. • What is the resolution of your data? • What is the date range of your available data? • If I made this query, how big would the response be? We hope to get answers when we distribute our ideas! (WARNING: we will stalk you if we don’t ;-) Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk

13. Conclusion Good… • Schema work progressing • Some sample implementations being produced Would be even better… • We need people to get involved and comment on the work that is talked about on the email list. • So, join the list and make Mark seem really quiet: • Send an email to majordomo@gridforum.orgwith a message body of "subscribe nm-wg" Mark Leese (Daresbury Laboratory) m.j.leese@dl.ac.uk