1 / 19

GENI Instrumentation and Measurement System - Schema

GENI Instrumentation and Measurement System - Schema. Martin Swany. Schema. Generally: model of objects and their relationships We should talk in terms of the general “schema” in this sense Renderings later XML schema SQL schema IPFIX JSON. I&M Schema Requirements.

dalton-whitehead
Download Presentation

GENI Instrumentation and Measurement System - Schema

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GENI Instrumentation and Measurement System -Schema Martin Swany

  2. Schema • Generally: model of objects and their relationships • We should talk in terms of the general “schema” in this sense • Renderings later • XML schema • SQL schema • IPFIX • JSON

  3. I&M Schema Requirements • Consistent basic representation of measurement and instrumentation data inside and outside of a slice • Flexibility of encoding and transport • timestamp and value or values • Expressive metadata • Easily extensible • Reusable components and models • Reasonable relationship to RSpec

  4. perfSONAR Model • perfSONAR is an internationally-adopted framework for instrumentation and measurement • Based on schemata and protocols defined in the *Grid Forum over the last 10 years • Recognized by the NSF as a key technology; recent NSF-funded workshop • Again, consider the model in the abstract rather than the XML encoding or RNC files

  5. perfSONAR Schema • Key Goals: Extensibility, Normalization, Readability • Break representation of performance measurements down into basic elements • Data and Metadata • Measurement Data • A set of of measurement events that have some value or values at a particular time • Measurement Metadata • The details about the set of measurement data

  6. Schema Normalization • Can simplify the database representation for many types of measurement data • While optimizations are possible, many measurement types can be viewed as one value measured over time • Assists Combination/Concatenation of metrics • Creating derived metrics • Normalization helps with inferring relationships between types of metrics

  7. Schema Basic Elements - Metadata • Subject (Noun) • The measured/tested entity • EventType (Verb) • What type of measurement or event occurred, or instrumented parameter was read • Characteristic, tool output, or generic event • Parameters (Adjectives and Adverbs) • How, or under what conditions, did this event occur?

  8. Schema Basic Elements - Data • Some sort of value - Datum • Existence of an event might point to the case where there no additional value • As in “Link up/down” or threshold events • Time • Is extensible since various representations are appropriate in different cases • E.g. UNIX timestamp vs NTP time

  9. Metadata Data A Message Message Message

  10. Metadata Data An Object Store Store

  11. A Data is Linked to a Metadata Metadata <id>someId</id> Data <metadataIdRef> someId </metadataIdRef>

  12. A Metadata may be linked to another Metadata <id>someId</id> Metadata <id>someOtherId</id> <metadataIdRef> someId </metadataIdRef>

  13. Schema Namespaces • Observation: all measurements have some sort of Data and Time • All measurements can be described by the Metadata identifying who, what and how • The specific structures of the Data and Metadata elements depend on the measurement • Approach: Consistently use Data and Metadata elements and vary the namespaces of the specific elements

  14. Schema Namespaces - 2 • We encode the measurement/event type in the namespace • And as a standalone element • Some components of the system can pass Data and Metadata elements through without understanding their specific structure • Allows and implementation to decide whether it supports a particular type of data or not • Allows validation based on extended (namespace-specific) schemata

  15. Schema Namespaces and Extensibility • One key to extensibility is the use of hierarchy with delegation • Similar to OIDs in the IETF management world • The OGF NM-WG defined a hierarchy of network characteristics • Good starting point • However, not all tools are cleanly mapped onto the Characteristic space • Often a matter of some debate

  16. Schema Namespaces and Extensibility • Organization-rooted tools namespace addresses this • Some top-level tools • ping, traceroute • Easy to add new tools in organization-specific namespaces • Performance Event Repository • Add a schema and get a URI • Add Java classes

  17. Linking Metadata • Metadata can be linked in two ways • Merge chaining allows for elements to be reused and a complete metadata can be built • Operation chaining requests or describes operations on data sets • Representation of data provenance A A A B B(A) B B

  18. Resources, Subjects, Topology • perfSONAR has a topology schema called UNIS – Unified Network Information Schema • Related to the control frameworks’ Resource Specification • Measurement and Instrumentation must be related to the resources themselves

  19. end

More Related