1 / 18

Measurement in the Internet

Measurement in the Internet. Paul Barford. University of Wisconsin - Madison. Spring, 2001. Why measure the Internet?. The size, growth, complexity and diversity of the Internet make it impossible to understand, manage, protect or provision without measurement.

roberttjackson
Download Presentation

Measurement in the Internet

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measurement in the Internet Paul Barford University of Wisconsin - Madison Spring, 2001

  2. Why measure the Internet? • The size, growth, complexity and diversity of the Internet make it impossible to understand, manage, protect or provision without measurement. • Prior measurement studies have taught us a great deal • Self-similarity of packet traffic • Ubiquity of power laws • Businesses use measurements to provision, manage and operate • Future developments will require more and/or different measurements

  3. What can be measured in the Internet? • Structure • Topology, routing, CDN’s, wireless, etc. • Traffic • Transport, end-to-end performance, etc. • Users and Applications • WWW, (x,my)Napster, Peer-to-Peer,Streaming, security, etc. • Failures • In all areas • Nefarious behavior • Pattern attacks, port scans

  4. Where are measurements made in the Internet? • For some measurements, this is obvious • Web logs • The goal for other measurements is to be “representative” • Various “Internet weather reports” • Placement of measurement nodes is not a well understood problem • More is better??

  5. How are Internet measurements made? • Passive methods • Application monitors (logs), packet monitors • Active methods • Probes, application simulation • Surveys • Significant infrastructure is always required • All methods present difficulties

  6. When must Internet measurements be made? • Diurnal traffic cycle • Time scales depend on “what” and “how” • Passive measurements are typically continuous • Can generate huge data sets • Many people will not allow access to their logs • Active measurements are typically discrete • Important characteristics can be missed • Probes can be filtered and/or detected

  7. Who is doing Internet Measurements? • Businesses do a great deal of measurement • What measurements are they taking and what do they do with their data? • Instrumentation for measurement-based research is relatively new • Developments over the past 12 years have been slow • 10’s of current studies (see CAIDA and SLAC pages for lists of these) • Most studies are not coordinated and relatively narrowly focused

  8. In the past… • Bellcore Ethernet packet traces – Leland et al. • High time resolution LAN data collected over 4 year period beginning in 1989 • Thorough analysis showed self-similar properties • NPD study – Paxson • Characterized routing and packet behavior in wide area • First installation of large measurement infrastructure

  9. Internet topology: Skitter Project • CAIDA • Internet topology measurement infrastructures • Traceroute studies focused on router/link discovery • Skitter: 17 sources, 54,000 destinations world wide • Data from Skitter is publically available • www.caida.org

  10. Skitter example – BGP paths

  11. Surveyor • Advanced Network Systems • Infrastructure for measuring one-way Internet latency, loss and routes between hosts • 61 sites world wide • GPS enabled • Closed platform (until recently) • www.advanced.org

  12. Surveyor example - delay

  13. Wide Area Web Measurements (WAWM) • Barford and Crovella (Wisconsin & Boston Univ.) • Application level measurement infrastructure • 11 clients (national and international) • GPS enabled • Combined passive and active measurements • Application of critical path analysis to TCP transactions • www.cs.wisc.edu/~pb

  14. Example of WAWM results File transfer delay for 500KB file between Denver and Boston

  15. National Internet Measurement Infrastructure (NIMI) • Paxson (ACIRI), Adams and Mathis (PSC) • Secure management platform for wide area measurements • Designed for general probe installation • Distributed client infrastructure • Principally academic sites (currently 41 world wide) • A number of projects are running on NIMI • www.ncne.nlanr.net

  16. Short Term Challenges of Internet Measurement • Developing methods for gathering more precise data • Current tools are frequently quite poor • Developing methods for analyzing measurement data more thoroughly • What we know about how things work is limited • Developing methods for understanding causes and effects across multiple domains

  17. The future – Global Internet Measurement (GIM) • Ubiquitous measurement capability • Embedded into the design of the Internet • Emphasis on extensible API’s for measurement • Analysis capability must be built in • Measurement quality and soundness • Data formats that enable aggregation that reflects higher level behaviors • Many difficulties • Management, security, privacy, heterogeneity, deployment, etc. • See Performance Evaluation vol. 864 www.elsevier.com/locate/peva

  18. Conclusion • Measurements are necessary for understanding Internet structure and behavior • Rule #1: Expect surprises! • Internet measurements are difficult • Rule #2: Expect the data to be “dirty” • Measurements are necessary for populating and validating any reasonable Internet models • Rule #3: Garbage-in-garbage out • Current measurement infrastructures can provide a great deal of data but fall short of the GIM goal

More Related