1 / 36

An Instrumentation and Measurement Architecture Supporting Multiple Control Monitoring Frameworks

An Instrumentation and Measurement Architecture Supporting Multiple Control Monitoring Frameworks . XXX Brazilian Symposium on Computer Networks and Distributed Systems III Workshop de Pesquisa Experimental da Internet do Futuro (WPEIF 2012) May 4th, 2012. Authors.

hang
Download Presentation

An Instrumentation and Measurement Architecture Supporting Multiple Control Monitoring Frameworks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Instrumentation and Measurement Architecture Supporting Multiple Control Monitoring Frameworks XXX BrazilianSymposiumon Computer Networks andDistributed Systems III Workshop de Pesquisa Experimental da Internet do Futuro (WPEIF 2012) May 4th, 2012

  2. Authors Marcelo M. Pinheiro14, Igor L. E. Macêdo1, Igor L. O. Souza1, Thiago S. Hohlenweger13, Paulo R. R. Leite1, Adriano L. Spínola1, Herbert Monteiro1, Raphael A. Dourado2, Leobino N. Sampaio4, José A. Suruagy Monteiro2andJoberto S. B. Martins11Salvador University (UNIFACS)2Federal University of Pernambuco (UFPE)3Federal Institute of Bahia (IFBA)4Federal University of Bahia (UFBA)

  3. Motivation • FIBRE-BR will possibly use three different control and monitoring frameworks in its nine islands • OFELIA Control Framework • cOntroland Management Framework (OMF) and • ProtoGENI • Each one takes a different approach in addressing I&M requirements and demands • Each CMF has its own monitoring capabilities • How to put all these together?

  4. Goal • An Instrumentation and Measurement Architecture Supporting Multiple Control Monitoring Frameworks • Our target is: • to provide, possibly, with a maximum reuse of the available CMFs I&M services over a new integrated and federated network structure; • To provide instrumentation and monitoring considering different I&M Services through FIBRE-BR (Monitoring Orchestration); • Multiple CMFs I&M data integration.

  5. FIBRE-BR I&M ARCHITECTURE

  6. 1 2 3 4 5

  7. Measurement Data Integration Point (MDIP) • conforms the collected data from the available CMFs to FIBRE-BR I&M standard format (NM-WG), representation and distribution (including visualization). • This service includes all measurement data processing related aspects such as, message format, message transport protocol and/or service, access privileges and common data storage or on-the-fly data distribution. 1

  8. The security and police service will use the global definitions implemented and controlled by the ClearingHouse (CH definition - is both an entity and a system consisting of software, operations, and policy to broker trust between federation partners.) 2 • Some services that are going to be used by the I&M Architecture: • Trust relationship (CA, SASL, etc) • Identity credentials • Integrated authentication/authorization • Federation level policies • Slice behavior • Data access policy • Policy enforcement • FIBRE-BR policy document

  9. The Orchestration and Configuration Services act on behalf of the users allowing them to configure, to define measurement points, and to orchestrate these measurement data collection facilities according to each individual CMF. 3 • Implementation initial ideas: • Use XML pub/sub messaging service, based on XMPP server • Currently supported by OMF • IMF @ GENI

  10. The I&M Portal main functionality is to provide a user friendly interface to control and access the measured data, according to a defined policy. 4 • Data visualization: • From real-time experiments • From data stored Persistent Data repository in each individual CMF (I&M perspective) • Verify privileges access • Available only toauthorized users

  11. The architecture has a storage strategy that allows users to retrieve data from their own or from others previous experiments, according to their access privileges. The persistent storage option is an experimenter decision that must comply with FIBRE-BR retention policy. • Implementation steps: • MDIP will be in charge of saving it persistently • Data retention policy • MySQL/RRD/SQLLite3/PostgreSQL database (access from I&M solution) • I&MStandard storage • Each CMF will keep its storage mechanism. I&M will, eventually, store it centrally or access it based on users demand and/or privilege • Logs storage 5

  12. OML Overview and OML-MDIP Proposal

  13. OML wasfirstdeveloped as themeasurementcomponentofOMF Today is a stand-aloneproject (independent) Shortly, it is a framework (set oflibraries) to collectandstoremeasurements OML Introduction

  14. OML MDIP consistsbasicallyofoneservice: OML MA (OML MeasurementArchive) • It’sresponsible for receivingandsendingtherequestsand responses • When it receivestherequests, the MA queries OML Server DB andthen uses standard perfSONARmessagestocommunicatewith FIBRE-BR I&M Services • The response is sent to the MDIP, in ordertobestoredpersistently in FIBRE-BR I&M repository (ifdemanded) OML MDIP Proposal(1)

  15. OML MDIP Proposal(2)

  16. FIBRE-BR I&M Architecture is designed to configure and collected data from different CMFs. It is intended to be an instrumentation and measurement evolutionary architecture There are a lot of challenges to overcome! Conclusions

  17. Backup slides

  18. ProtoGENI I&M Architectureand Tools

  19. Introduction • There are several GENI I&M related projects compatible with ProtoGENI. The most important ones are: • INSTOOLS • LAMP • OneTimeMeasure • S3Monitor • Some of them are complimentary and will later be integrated to compose a GENI Integrated I&M Framework

  20. Instrumentation Tools (INSTOOLS)

  21. Overview • INSTOOLS’ high-levelgoal: • Make it easy for users to see what is going on in their experiment – i.e., make it trivial to monitor a slice • What can INSTOOLS measure? • Note that INSTOOLS is concerned only about passive measurements • INSTOOLS’ philosophy • Don’t reinvent the wheel

  22. Architecture (overview) MC Portal: single interface for allMCs in a slice Measurement Plane connections Data Plane connections Experimenter Slice’s nodes (instrumentizedtoact also as MPs) MeasurementController (MC) = GENI MAP+MC Automaticallydeployedby INSTOOLS (atleastone per aggregate)

  23. INSTOOLS Web Portal

  24. LAMP

  25. Overview • LAMP stands for “Leveraging and Abstracting Measurements with perfSONAR” • The main goal is to “create an instrumentation and measurement system, based on perfSONAR, for use by experimenters on ProtoGENI” • Which tools does LAMP support? • OWAMP, BWCTL, Ganglia,PingER, NTP; ps-BUOY MA; etc • So... what’sthedifferencebetween LAMP andpS-PSToolkit? • LAMP adapted perfSONAR-PS software suite to recognize GENI’s Authentication and Authorization model and infrastructure • Added Ganglia as a host monitoring solution • Added distributed configuration through annotations in the topology stored in UNIS • Theseannotationsmake it easy for users to save their slice configuration and load it at a different slice

  26. Architecture (a broaderview) ProtoGENI Node with LAMP tools UNIS Node with LAMP tools + LAMP Portal enabled UNIS keepsinformationaboutALLslices Note that it is possible to have more than one node running the Web Portal This node hasonly “Measurement Plane interfaces” Slicen Slice 2 Slice 1 Notevery node hastobe “instrumentized”

  27. LAMP Portal Each node has as set of (possibledifferent) measurement tools enabled

  28. OneTimeMeasureand S3Monitor

  29. OneTimeMeasureand S3Monitor • OnTimeMeasure • Is “an on-demand measurement service used in forecasting, anomaly detection, and fault-location diagnosis in GENI experiments and GENI operations.” • Canbeintegratedwith INSTOOLS • S3Monitor • Has a flexible design that allows easy “plug in” of new network measurement tools • Basedon (Scalable Sensing Service for PlanetLab) • Focused on measurement of large networked systems • Already integrated with INSTOOLS

  30. Integrating with FIBRE-BR I&M architecture

  31. INSTOOLS MDIP Fibre I&M Architecture ProtoGENI Visualization Portal Slice (INSTOOLS) 4 ProtoGENI MDIP INSTOOLS MeasController RRD Collector 3 1 2 SQL Collector RRD SQL DB FIBRE I&M Data Repository MC collects measurement data from the MPs MDIP (through his collectors) collects measurement data, makes any necessary format adjustments and Stores the data in the permanent repository (if demanded) Measurement data can be accessed through the Visualization Portal

  32. LAMP MDIP ProtoGENI Fibre I&M Architecture Visualization Portal Slice ProtoGENI MDIP UNIS MA-specific defs LAMP Web Portal 1 MP SNMP MA NMWG GenericCollector 3 ps-BUOY MA 2 Ganglia MA MP PingER MA FIBRE I&M Data Repository Fetch experiment description from UNIS Start copying measurement data from MAs Stores the data in the permanent repository (if demanded)

  33. OFELIA I&M Architecture Igor Luiz (UNIFACS)

  34. NOWADAYS Backend Infrastructure Monitoring (servers, switches, links) - ZENOSS (not integrated to OCF) OFELIA´s experiment monitoring: - Still under discussion and development OFELIA I&M STATUS

  35. FIBRE I&M possiblealternatives for OFELIA CMF are (in discussion): • Focus onFlowVisorand OF-relatedbasicmeasurementsparameters (pragmaticapproach) • Incorporatecurrent OFELIA monitoringdevelopments (needtoidentifythem) • Alignwith OFELIA currentdevelopments OFELIA & FIBRE I&M • OFELIA I&M technicalpossibilities/discussions: • OML • sFlow • perfSONAR • LAMP • ... !?

More Related