E2E.piPEs Update: Overview and Deployment Status

1. Internet2 E2E piPEs Update Eric L. Boyd

2. Overview • E2E piPEs Overview • What is piPEs? • Goals • E2E piPEs Measurement Infrastructure • LA  CERN Measurement Demo • piPEs Status • Where are we? What’s available? • What’s next? What can you do?

3. Internet2 E2E piPEs • Project: End-to-End Performance Initiative Performance Environment System (E2E piPEs) • Approach: Collaborative project combining the best work of many organizations, including DANTE/GEANT, Daresbury, EGEE, GGF NMWG, NLANR/DAST, UCL, Georgia Tech, etc.

4. Internet2 E2E piPEs Goals • Enable end-users & network operators to: • determine E2E performance capabilities • locate E2E problems • contact the right person to get an E2E problem resolved. • Enable remote initiation of partial path performance tests • Make partial path performance data publicly available • Interoperable with other performance measurement frameworks

5. Measurement Infrastructure Components

6. Sample piPEs Deployment

7. Measurement Software Components

8. Europe Hawaii OSU NC State UCSD 1) Abilene Backbone Deployment (Complete) 2) Hawaii Campus Deployment (Complete) 3) Campus, European, APAN Deployment (In Progress) piPEs Deployment

9. Overview • E2E piPEs Overview • What is piPEs? • Goals • E2E piPEs Measurement Infrastructure • LA  CERN Measurement Demo • piPEs Status • Where are we? What’s available? • What’s next? What can you do?

10. American/European Collaboration Goals • Awareness of ongoing Measurement Framework Efforts / Sharing of Ideas (Good / Not Sufficient) • Interoperable Measurement Frameworks (Minimum) • Common means of data extraction • Partial path analysis possible along transatlantic paths • Open Source Shared Development (Possibility, In Whole or In Part) • End-to-end partial path analysis for transatlantic research communities • VLBI: Onsala, Sweden  Haystack, Mass. • HENP: CERN, Switzerland  Caltech, Calif.

11. American/European Demonstration Goals • Demonstrate ability to do partial path analysis between “Caltech” (Los Angeles Abilene router) and CERN. • Demonstrate ability to do partial path analysis involving nodes in the GEANT network. • Compare and contrast measurement of a “lightpath” versus a normal IP path. • Demonstrate interoperability of piPEs and analysis tools such as Advisor and MonALISA

12. Demonstration Details • Path 1: Default route between LA and CERN is across Abilene to Chicago, then across Datatag circuit to CERN • Path 2: Announced addresses so that route between LA and CERN traverses GEANT via London node • Path 3: “Lightpath” (discussed earlier by Rick Summerhill) • Each measurement “node” consists of a BWCTL box and an OWAMP box “next to” the router.

13. All Roads Lead to Geneva

14. Results • BWCTL: http://abilene.internet2.edu/ami/bwctl_status_eu.cgi/BW/14123130651515289600_14124243902743445504 • OWAMP: http://abilene.internet2.edu/ami/owamp_status_eu.cgi/14123130651515289600_14124243902743445504 • MONALISA • NLANR Advisor

15. Insights (1) • Even with shared source and a single team of developer-installers, inter-administrative domain coordination is difficult. • Struggled with basics of multiple paths. • IP addresses, host configuration, software (support source addresses, etc.) • Struggled with cross-domain administrative coordination issues. • AA (accounts), routes, port filters, MTUs, etc. • Struggled with performance tuning measurement nodes. • host tuning, asymmetric routing, MTUs

16. Insights (2) • Connectivity takes a large amount of coordination and effort; performance takes even more of the same. • Current measurement approaches have limited visibility into “lightpaths.” • Having hosts participate in the measurement is one possible solution.

17. Insights (3) • Consider interaction with security; lack of end-to-end transparency is problematic. • Security filters are set up based on expected traffic patterns • Measurement nodes create new traffic • Lightpaths bypass expected ingress points

18. Overview • E2E piPEs Overview • What is piPEs? • Goals • E2E piPEs Measurement Infrastructure • LA  CERN Measurement Demo • piPEs Status • Where are we? What’s available? • What’s next? What can you do?

19. piPEs Status • Tools • BWCTL (released) Traceroute (ongoing) • OWAMP (released) NDT/piPEs Integration (ongoing) • Measurement Framework • Abilene Measurement Infrastructure (deployed) • Measurement Infrastructure “Glue” (prototype, ongoing) • Edge-to-Middle Testing (deployed) • Outreach • Application Communities (HENP, VLBI, ongoing) • Establishing Measurement Domains (ITECs, ongoing) • Collaboration Partners (DANTE, ESnet, APAN, ongoing) • Standards Integration (GGF NMWG [Advisor, MonALISA], IETF, ongoing) • Measurement Framework Federation

20. BWCTL (Jeff Boote)http://e2epi.internet2.edu/bwctl

21. OWAMP (Jeff Boote)http://e2epi.internet2.edu/owamp

22. NDT (Rich Carlson) • Network Diagnostic Tester • Developed at Argonne National Lab • Ongoing integration into piPEs framework • Redirects from well-known host to “nearest” measurement node • Detects common performance problems in the “first mile” (edge to campus DMZ) • In deployment on Abilene: • http://ndt-seattle.abilene.ucaid.edu:7123

23. Abilene Measurement Domain • Part of the Abilene Observatory: http://abilene.internet2.edu/observatory • Regularly scheduled OWAMP (1-way latency) and BWCTL (Iperf wrapper) Tests • Web pages displaying: • Latest results http://abilene.internet2.edu/ami/bwctl_status.cgi/TCP/now “Weathermap” http://abilene.internet2.edu/ami/bwctl_status_map.cgi/TCP/now • Worst 10 Performing Links http://abilene.internet2.edu/ami/bwctl_worst_case.cgi/TCP/now • Data available via web service: http://abilene.internet2.edu/ami/webservices.html

24. AMI Web Service (W. Matthews)http://abilene.internet2.edu/ami/webservices.html my %request = ( networkCharacteristic => "path.delay.oneway", subject => { source => { address => { name => "ATLA", }, }, destination => { address => { name => "LOSA", }, }, type => "IPv4", }, startTime => ”2004-02-01T00:00::00-5:00", endTime => ”2004-02-01T23:59:59-5:00", ); my $webService = SOAP::Lite -> uri('http://abilene.internet2.edu/PIPES') -> proxy('http://username:password@abilene.internet2.edu/ami/services/soap_server.cgi') -> NetworkMeasurementSet(\%request) -> result; Wrap the request up in an object Send it to the webservice

25. Test from the Edge to the Middle • Divide and conquer: Partial Path Analysis • Install OWAMP and / or BWCTL • Obtain a BWCTL key here: • http://e2epi.internet2.edu/pipes/ami/bwctl/ • http://e2epi.internet2.edu/pipes/ami/owamp/ • Begin testing!

26. Outreach • Establishing Application Communities • HENP, VLBI • Establishing Measurement Domains • ITECs, CERN, GEANT • Collaboration Partners • DANTE, ESnet, APAN • Standards Integration • GGF NMWG, IETF  NLANR / DAST Advisor, MonALISA

27. Measurement Infrastructure Federation • Why a Federation? • Multiple measurement frameworks in existence and under development (piPEs, NLANR Advisor, NLANR AMP, etc.). • No static “best practice” measurement framework is likely to emerge, given academics being academics. • Future measurement frameworks can build on shoulders of current efforts, not feet. • Performance Measurement Architecture Workshop (NSF Grant # ANI-0314723) • Matt Zekauskas

28. Measurement Infrastructure Federation Interfaces

29. Using piPEs • Download, try out BWCTL, OWAMP • Test Edge  Middle • Set up Measurement Beacons • Identify machines & locations • Determine policies and limitations • Install BWCTL, OWAMP daemons • (Future): Set up measurement domain or application community

30. Summary • We can do partial path analysis, although making sense of the results is still a big issue. • We can speak the same measurement language, although it’s still evolving. • We are working together in growing numbers, but we need critical mass (become de facto standard). • We need to be able to find each other. • We need to be able to verify each other’s identity.

32. Extra Slides

33. Collection Today: Iperf (Throughput) OWAMP (1-Way Latency, Loss) SNMP Data Anonymized Netflow Data Per Sender, Per Receiver, Per Node Pair IPv4 and IPv6 Collection in the Future NTP (Data) Traceroute BGP Data First Mile Analysis Correlation Today: “Worst 10” Throughputs “Worst 10” Latencies Correlation in the Future: 99th Percentile Throughput over Time Throughput/Loss for all E2E paths using a specific link Commonalities among first mile analyzers Sum of Partial Paths vs. Whole Path Data Collection / Correlation

34. Analysis Today: Throughput over Time Latency over Time Loss over Time Worrisome Tests? (Any bad apples in “Worst Ten”?) “Not the Network” (If “Worst Ten” is good enough) Analysis in the Future: Latency vs. Loss How good is the network? Do common first mile problems exist? Does a link have problems that only manifest in the long-haul? Is the network delivering the performance required by a funded project? Data Analysis

35. Discovery in the Future: Where are the measurement nodes corresponding to a specific node? Where are the test results for a specific partial path? Interoperability in the Future: Can I have a test within or to another measurement framework? Can I have a measurement result from within or to another measurement framework? Data Discovery / Interoperability

36. Europe Hawaii OSU UCSD 1) Abilene Backbone Deployment (Complete) 2) Hawaii Campus Deployment (Complete) piPEs Deployment NC State APAN 3) Campus, European, APAN Deployment (In Progress)