Update on GGF Measurement Activities

1. Update on GGF Measurement Activities Bruce Lowekamp The College of William and Mary

2. My Research • William and Mary Computer Science Department • 14 Faculty • 45 PhD students • 60 Masters students • lots of good undergraduates • Research • Remos: SNMP-based topology and utilization for distributed apps • Wren: leveraging topology and passive measurements for scalable grid network performation measurement • Optimistic grid computing: fine-grained apps on a grid • GGF Network Measurements Working Group

3. Passive Network Measurement • When an application is running, use passive measurements. When not, use active probes. • Controlled by monitoring system, knows when measurements needed. • Conversion between measurements important.

4. Importance of Topology in Grids • No one rule governs performance • Real users (and system owners) make bad choices • Grid applications must optimize performance for these environments. • Can we exploit topology knowledge for better • measurements? • application performance?

5. Outline • GGF’s perspective on network measurement • GMA: Grid Monitoring Architecture • DAMED: Top-N Events • NMWG: Network Characteristics Hierarchy

6. R ? R R ? R R R R R R network dispersed users R ? ? R R R R R R R R R R VO-A VO-B GGF Perspective • Users of measurements • Application designers • Runtime system designers • Many users, many environments • Grid applications must be flexible, portable

7. Information Portability • Information must be portable • Each AS/VO may pick its own measurement system • Parts of network aren’t measured • Different parameters • Goal: Application runs, unaware of environment • Information from multiple measurement systems • Should not have to support 10 different performance models

8. GGF Projects • GMA: Grid Monitoring Architecture • What components do we need? • DAMED: Discovery And Monitoring Event Description • What are the Top-N events we need to support RIGHT NOW? • NMWG: Network Measurements • What does “bandwidth” mean anyway? • Components of this global information service • Boils down to schemas and protocols

9. Existing Pieces • Many of these components already exist or are in progress: • instrumentation tools • Pablo (UIUC), NetLogger (LBNL), log4j (apache), web100, SNMP, etc. • host and network sensors • too many to list • sensor management tools • JAMM (LBNL) • event publication service • MDS (Globus), NWS (UCSB), R-GMA (RAL),CODE (NASA AMES), Remos(CMU) • event archive service • netarchd (LBNL), NWS (UCSB) • event analysis and visualization tools • lots, but most only work for specific types of events: • NetLogger nlv (LBNL), Probe (Stazi), Autopilot (UIUC), etc. • BUT, all use different event formats and protocols! • no interoperability

10. Event Publication • To handle potentially huge amounts of event data requires an event publication and subscription service that is: • flexible • highly scalable • provides near real-time access to monitoring data • The Global Grid Forum (GGF) (www.gridforum.org) has defined the “Grid Monitoring Architecture” (GMA), for this purpose. • Several GMA implementations have started to appear • A great deal of work remains to define standard event schemas and event dictionaries for the GMA.

11. GMA Terminology and Architecture • (Performance) Event: • Typed collection of data with a specific structure • Producer Interface: • makes performance data (events) available • Consumer Interface: • receives performance data (events) • Directory Service: • supports information publication and discovery • must be distributed and/or replicated

12. DAMED WG • Discovery And Monitoring Event Description Working Group • Chairs • Jennifer Schopf, ANL • James Magowan, IBM • Top-N Metrics

13. DAMED Charter • Define a basic set of monitoring event descriptions • information (attributes) associated with a particular data element • conventions for the representation of the value associated with it. • Develop standard representations of the most widely used measurement values (the "top N".) • Emergence of a set of conventions and recommendations that will ease the task of defining richer, domain-specific schemas • Damed if we do • Not everyone will be happy • Damed if we don’t • Never reach our goal of seamless interoperability of grids (one big grid e.g. internet)

14. DAMED Terminology • Events • Event Target • Event Type • Event Name = Target.Type • network.link • delay.TCP • network.link.delay.TCP

15. Target Types • Targets used in Top-N Events • Host: IP • Process: IP, PID • Disk Partition: /home • Network Link: IP {port},IP {port} • Software: String • Scheduler: IP, String • Not necessarily hierarchical

16. Event Types • Top-N • CPU Load • System uptime • Disk size • Disk used • TCP available bandwidth • Ping RTT • Traceroute number of hops • Running software status • Packet Loss • Available Memory • Host Architecture • Host OS • Physical Memory

18. NMWG • Network Measurements Working Group • Chairs: • Brian Tierney (LBNL) • Bruce Lowekamp (W&M) • Richard Hughes-Jones (Manchester) • Goal: • Portability of network measurements • Steps: • Define hierarchy of measurements • Establish mapping of tools<->measurements • Conversion between measurements of same type

19. Characteristics Hierarchy • Ultimate Goal: Portability of Measurements • Many APIs • Many tools • Natural Grid Development Process • More measurement systems • More measurement tools • More cooperation • More shared deployed infrastructure • Middleware must be able to determine what network performance information is measuring. • How do we share measurement information without discouraging development of new APIs and tools?

20. How the Nomenclature Helps • Need to classify measurements • What does it measure? Sometimes more important than how. • Not necessarily a new schema • Should be a good schema for network measurements • Not all systems are/should be organized this way • Can be used as annotation in any schema. • Goal is an agreed-upon classification of measurements taken, to allow both current and future measurement methodologies to classify their observations to maximize their portability.

21. Representing a Measurement • A measurement is represented by two elements: • Characteristic What is being measured. Bandwidth, latency, etc. • Network Entity The part of the network described by the measurement Link, path, host, etc.

22. Terminology • Network Characteristics • Intrinsic properties of a portion of the network that are related to its performance and reliability • Measurement Methodologies • Means and methods of measuring those characteristics • Observation • An instance of information obtained by applying a measurement methodology. • Note on IETF IPPM RFC2330 • Compatible where possible, but metrics means many things. • Guiding principle: clear meanings, follow standards where defined.

23. Network Characteristics • “Intrinsic Property” • Property itself, not an observation • Unrelated to how measurement is made • Not a particular number • Packet Loss • Fraction of traffic • Loss patterns • Traffic profile

24. Measurement Methodology • Technique for recording or estimating a characteristic • Two approaches: • Raw: measuring actual characteristic • Derived: aggregate or estimate from other characteristics • Round trip delay • ping • TCP transmit/ACK pair • two one-way delay measurements • link propagation and queue length data

25. Observations • Singleton • Smallest possible observation • Sample • Several singletons together • Statistical • Derived from a sample by calculating a statistic • Timestamps, and ranges, are issues with each observation

26. Network Entities • Attributes must be included. • Nodes and paths can be physical or functional.

27. Describing Topology • Two different types of topology • Physical: Actual links and nodes • Functional: Derived closeness • Attributes define the Path or Node • Multiple Topologies are Superimposed over physical network

28. Describing Topology • Paths: Path data follows from source to destination • Unidirectional in most cases • Paths (including hops) may be made of components • Nodes: Hosts and Internal nodes • Physical and Functional graphs not disjoint at edges

29. Characteristics Overview

30. Relationship Between Measurements • Can we develop systems that use whatever information is available? • iperf • pathload • QoS support • Need to be able to request measurement of particular characteristic, without regard to what sub-characteristic or tool is used to return the result. • Convert loss pattern to loss rate. • Traffic profile to utilization fraction.

31. Characterization of Tools • Goal of hierarchy is to make measurements portable. • First step is to agree on what characteristic tools measure. • Some tools measure multiple characteristics, depending on parameters. • Many lists of tools, including E2EPI, our goal is to annotate these lists and produce hierarchy with multiple views.

32. NMWG Upcoming Work • Taxonomy is nice, but exchanging real data requires a schema, with values for attributes and parameters. • Two steps: • Map tools to taxonomy • Produce schema • Schema step is needed to reach goal of portability.Participants including DAMED members.

33. Summary of GGF Activities • Focus on two aspects: • System interoperability • Measurement portability • GMA completed • DAMED finishing up Top-N documents • NMWG characteristics hierarchy near release • Need schema to put components together • Portions contributed by: Jennifer Schopf (ANL)James Magowan (IBM), Brian Tierney (LBNL), and Dan Gunter (LBNL)