Is your Service Available? or Common Network Metrics Nevil Brownlee, CAIDA NANOG 19, Albuquerque, June 2000

1. Is your Service Available? orCommon Network Metrics Nevil Brownlee, CAIDANANOG 19, Albuquerque, June 2000

2. Overview • CAIDA Metrics Working Group • Co-chairs from the networking industry • Sue Moon (SprintLabs) • Brett Watson (MFN/Abovenet) • Measurement FAQ • Service Definitions • Common Metrics • Availability NANOG 19, June 2000 Nevil Brownlee

3. Metrics WG Goals • Education • Publish ‘Measurement FAQ’ • Publish ‘Metrics and Measurement Survey’ • Service Metrics • Define metrics for new / emerging services • Recommend a ‘Service Measurement Toolkit,’encourage implementors • Publish revised ‘Measurement Requirements for Hardware/Software Vendors’ • User / customer participation needed ! NANOG 19, June 2000 Nevil Brownlee

4. FAQ Contents (1) • Target audience • Corporate users, smaller providers, hosting service users • Generally Accepted Terms • Networking, types of service, faults and failures • Measurement Topics • Active vs passive, one point vs many, sampling, statistics NANOG 19, June 2000 Nevil Brownlee

5. FAQ Contents (2) • The Most Common metrics • Latency, packet loss, throughput, link utilisation, availability • Common Measurement Tools • ping, traceroute, SNMP, flow measurementapplication monitoring, visualization • Comparing Service Offerings • Provider ‘net status’ pages • Internet ‘weather maps’ • Rating services NANOG 19, June 2000 Nevil Brownlee

6. Defining Network Service • Service definitions (SLAs) on the Web • Many providers publish these, e.g. AT&T, PSInet, UUNET and MCI WorldCom • They describe service offered to customers • We’re only interested in describing the service, not in contractual aspects • Metrics used in the service descriptions are often poorly defined NANOG 19, June 2000 Nevil Brownlee

7. Service Example: UUNET (1) • Network Quality • Average monthly latency of no more than 85ms roundtrip within UUNET's network in North America and of no more than 120ms between New York and UUNET's international gateway hub in London • Comments • Restricted to provider’s own network • Latency not defined • Nothing said about packet loss % NANOG 19, June 2000 Nevil Brownlee

8. Service Example: UUNET (2) • Service Quality • 100 percent availability that covers the UUNET backbone and the UUNET-ordered customer access circuit. • Scheduled maintenance .. will take place .. Tuesdays and Thursdays with at least 48 hours advance notice • Comments • 1 minute per week = 0.01% • Does ‘available’ time include maintenance? • ‘100% availability’ - but what is availability? NANOG 19, June 2000 Nevil Brownlee

9. ISP Network Report Pages • Many providers publish these • Sean Donelan email thread lists 13, e.g. Abovenet, AT&T, C&W, UUNET, ELI, Jet Net .. • Amount of detail varies • Outage information, NOC contacts • Latency, packet loss matrices (or averages) • Such ‘overall’ reports don’t say much about performance as seen from your network • You need some measurements at your site NANOG 19, June 2000 Nevil Brownlee

10. Common Metrics • Throughput, link utilisation • Commonly measured with SNMP, RRDtool • Latency, packet loss • Latency is round-trip, transit + server delay • Commonly measured with ping • Availability • WG definition based on IETF IP Performance Measurement (IPPM) connectivity metrics • Need to specify what’s available, how to measure it, and what values are acceptable NANOG 19, June 2000 Nevil Brownlee

11. Levels of Availability Service availability: being able to send packets for a specified service - say WWW request packets - to a particular Internet host, and to receive answering packets Host availability: being able to send packets, say ping packets, to a particular Internet host, and to receive answering packets Network availability: being able to send packets from your network to the Internet, and to receive answering packets NANOG 19, June 2000 Nevil Brownlee

12. Measuring Availability Web service availability test: download specified pages from target web server using web browser, measure latency, packet loss and throughput Host availability test: ping the target host, having made sure that it will respond to ICMP packets Network availability test: traceroute to the target host, so as to determine whether there is connectivity to the target network NANOG 19, June 2000 Nevil Brownlee

13. Defining Availability • Requires negotiation between provider and customer • What services are important? • What performance level is acceptable? • Be realistic! • Providers only control their own networks • Some packet loss is inevitable • Measurements are important • Both sides should work together on this • Make some ‘baseline’ measurements NANOG 19, June 2000 Nevil Brownlee

14. Conclusion • The CAIDA Metrics WG has begun by producing its ‘Measurement FAQ’, which provides background material on many measurement topics • The FAQ attempts a new definition of Service Availability - the Metrics WG needs feedback on this ! • The WG seeks input for its other goals, especially for defining new metrics NANOG 19, June 2000 Nevil Brownlee

15. More Information (CAIDA) • Metrics WG (FAQ, mailing list) www.caida.org/outreach/metricswg • FAQ Contributors Cindy Bickerstaff, Carter Bullard, Les Cottrell, Sean Donelan, Dave O’Leary, Brett Watson, .. • CAIDA tools taxonomy www.caida.org/tools NANOG 19, June 2000 Nevil Brownlee

16. More Information (Net Status) • Provider web pages are listed in FAQ [ISP_SERV] service definitions [ISP_REPT] network performance reports • NOC pages (search ‘xxx network status’) www.pictureview.com/support/PVTS2.html www.psinet.com/netstatus/ www.sprintlink.net/netstat.html • Network performance pages stats.sjc.above.net/traffic/ ipnetwork.bgtmo.ip.att.net traffic.cwusa.com/index.html NANOG 19, June 2000 Nevil Brownlee