Internet2 Detective: Building a Dynamic Performance Measurement Framework

1. piPEfittersSalt Lake City Jt Techs (Feb 05) Jeff Boote - Internet2

2. Agenda • Current activities and Tools update • High level framework description • Summary: Next Steps

3. piPEs • BWCTL • Stable - fair amount of interest • Recent requests for additional functionality (parallel streams, multicast) • OWAMP • Significant changes to specification. IETF working group last call completed • New version of implementation forthcoming to reflect the changes (BEFORE SMM!)

4. piPEs • NDT • Redirection to closest NDT server within a group of servers • Funded to significantly improve understanding and detection of duplex mismatch problems (NIH/NLM Grant) • PMP registry http://e2epi.internet2.edu/pipes/pmp/pmp-dir.html

5. piPEs • Internet2 Detective • Strategic investment: Gateway for naïve entrance to advanced services like Shibboleth and Pipes • Evaluating future development using SURFnet Detective platform

6. Current work: Joint Development with Geant2 (JRA-1) • Rather than build two separate interoperable measurement frameworks, why not jointly develop a single measurement framework? • Steps: • Agree to joint open source development √ • General Framework Design √ • Prototype (Summer ’05) • Detailed Design • Implementation • Seek participation from NRENs & campuses, particularly Internet2 & ESnet members • Thrice weekly conference calls • Very active mailing list • 2-3 face-to-face meetings per year

7. Architecture Refinement • Review of existing systems • Insights based upon Abilene prototype framework, DANTE’s perfmonit and IPPM experiences • New insights gained from inter-domain framework test experience (lightpath measurements, Abilene/ESnet, etc) • Additional use cases and experience of collaborators • Internet2, GÉANT2 JRA1, GGF NMWG

8. Design Goals • Dynamic, self-organizing characteristics identical to that of the network as a whole • Recognize and facilitate the ability of independent network entities to set policies and limits on the use of measurement resources locally • Encourage and facilitate the use of measurement resources by users interested in network paths that traverse remote administrative domains • Facilitate the widespread adoption of new performance tools in a broad, E2E framework • Allow framework to evolve over time

9. Architecture Proposal • Services-Based Measurement Framework for Building Dynamic, Self-Organizing Performance Communities • In a simple scenario, each domain consists of a set of services. All services are well defined and independent • Services within a domain represent the domain with the help of Authentication and Authorization – they respond to requests only if the Authentication service of the domain has authenticated the user and the policy of the given service authorizes it

10. Basic Services • Lookup • Authentication • Measurement Point • Resource Protector (Authorization) • Measurement Archive • Aggregation • Topology

11. Test Request (Initialization)

12. Lookup Service • Initial discovery • Multicast / Anycast • Well known servers • Required servers (by administrative configuration) • Previously detected servers (organized in a P2P network – lookup services find out about other lookup services…

13. Lookup Service (II) • Lookup is not simply by name • Type (type of measurement, type of service) • Community • Network path (proximity information from Topology) • Organization • Type of authentication required • Other… • Response contains • Contact information • Available services • Authentication required • Other…

14. Authentication Service • Registers with lookup • Client requests “kind” of authentication token based on lookup results • Authentication grants time-limited token used to request service • Attribute service created to protect privacy and support role-based authorization • Allow new measurement points to be created as easily as possible • Allow new data consumers access as easily as possible

15. Authenticated Service Request(see notes for slide)

16. Federated Identity Service Request(see notes for slide)

17. Measurement Point • Service to wrap measurement tools • Interacts with resource protectors to protect shared resources • Registers with lookup service and specifies the authentication credentials required to interact • Registers with lookup service to indicate types of tests it can perform • Accepts requests for tests

18. Client Process Flow • Discovery. • Find lookup servers. • Use lookup servers to find measurement tool (MPs) for a given problem. (On correct path, with acceptable authentication requirements, with acceptable tools/measurements.). • Authentication. • Authenticate to correct auth servers that are needed for desired MPs. • Test execution. • Implement subscriber to accept results. • Make test requests presenting credentials and reference to subscriber interface for returned data.

19. Resource Protector • Enables centralizing of resource allocation (not globally - this is within spheres of administrative control) • Multiple measurement points interact with a given resource protector to limit the shared resources • Resource protectors can be chained to control aggregations of shared resources

20. Flow for Measurement Request

21. Resources Protectors

22. Measurement Archive • Subscribes to some set of data – either from a measurement point or from an aggregation service • May publish the derived data sets

23. Transformation Service • Pipelines data between other components in the framework • Subscribes and Publishes data • Provides: • Aggregation • Correlation • Caching • Duplication • Filtering • Translation • Event generation • Data analysis

24. Topology • Specific type of transformation service (correlation) • Network topology information is necessary for measurement system optimization • Creates overviews/”maps” to illustrate network • Collects raw data from measurement points and pushes topology information into Measurement Archives • Lookup service makes queries to MAs to deduce topological adjacencies. (allows location based queries to lookup service)

25. General Framework Next Steps • Architecture continuing to be refined • Architecture validation • Detailed use-case flow descriptions • Interfaces • Prototypes • Jointly developed, services-based, measurement framework prototype by Summer ‘05

26. Summary: Next Steps • Open Source Shared Development • Sourceforge-based Sub-Projects • Modified Berkeley Licensing • Common Service-based Architecture • Architecture spans superset of deployment use cases • ~Quarterly face-to-face meetings • ~Thrice-Weekly phone conferences • Split development according to interest, resources

27. Questions? • Are you interested in participating? • Contact eboyd@internet2.edu • Talks of Interest • Measurement SIG • Tuesday, 7 PM, Marriot Ballroom 1 • Transport Track session for a introduction into efforts of ad hoc working group aiming to get benefit of kernel level congestion control algorithm improvements into a user level bulk FTP tool • Wednesday, 10:25 AM, Saltair • GGF NMWG meeting for a detailed discussion of a common measurement schema • Wednesday, 2:00 PM, Willow Room, 4th Floor