1 / 14

Distributed Network Monitoring in the Wisconsin Advanced Internet Lab

Distributed Network Monitoring in the Wisconsin Advanced Internet Lab. Paul Barford Computer Science Department University of Wisconsin – Madison Spring, 2002. Motivation. Many applications that run over the Internet have minimum performance requirements

giza
Download Presentation

Distributed Network Monitoring in the Wisconsin Advanced Internet Lab

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Network Monitoring in the Wisconsin Advanced Internet Lab Paul Barford Computer Science Department University of Wisconsin – Madison Spring, 2002

  2. Motivation • Many applications that run over the Internet have minimum performance requirements • The network is one of the two possible sources of poor performance • Wide area network behavior is unpredictable • IP networks are best effort • Constant change is normal • Quality of service capability is not widely deployed • Will it ever be available? pb@cs.wisc.edu

  3. Monitoring is a First Step • Accurate monitoring of network state can enable application adaptivity and improved network management • Data provides basis for improved models and protocols • There are many challenges in network monitoring • All features of the Internet make monitoring difficult • When, where, what, how… • Today’s focus • Network monitoring efforts at Wisconsin • Combining monitoring and analysis to understand network traffic anomalies pb@cs.wisc.edu

  4. The Wisconsin Advanced Internet Lab • Next generation environment for network research • Our focus: performance, management, security • Platform for testbeds: storage, grid computing , … • Internal environment • Instances of end-to-end-through-core Internet paths • External environment • Measurement nodes deployed across the Internet pb@cs.wisc.edu

  5. WAIL’s External Environment • Existing infrastructure • WAWM systems (10) • Surveyor systems (60) • Partnership with Advanced Systems • NIMI systems (45) • Partnership with PCS and ICIR • Condor/Grid Infrastructures • Prototype system is under development • Passive flow measurements • FlowScan data from UW, Internet2, others(?) pb@cs.wisc.edu

  6. WAIL’s Internal Environment • Complement to external facilities • Hands-on test bed which creates paths identical to those in the Internet from end-to-end-through-core • Variety of highly configurable equipment • Why do we need an internal lab? • Enables instrumentation and measurement of entire end-to-end system • Enables new systems and protocols to be implemented in places where access is not possible in wide area • Vision of internal lab: New means for doing network research • Status: Significant commitment from industry partners (Cisco, EMC, Fujitsu) and the university – rev. 1.0 by 5/1/02 pb@cs.wisc.edu

  7. Distributed Anomaly Detection • Motivation: Anomaly detection and identification is an important task for network operators • Operators typically monitor by eye using SNMP or IP flows • Simple thresholding is ineffective • Some anomalies are obvious, other are not • Focus: Characterize and develop distributed means for detecting classes of anomalies • Network outages, Flash crowds, Attacks, Measurement failures • Approach: Use statistical and wavelet techniques to analyze anomalies from IP flow and SNMP data from UW and other sites • Implications: Tools and infrastructure which quickly and accurately identify and adapt to traffic anomalies pb@cs.wisc.edu

  8. Characteristics of “Normal” traffic pb@cs.wisc.edu

  9. Our Approach to Analysis • Analyze examples of each type of anomaly via statistics, time series and wavelets (our initial focus) • Wavelets provide a means for describing time series data that considers both frequency and scale • Particularly useful for characterizing data with sharp spikes and discontinuities • More robust than Fourier analysis which only shows what frequencies exist in a signal • Tricky to determine which wavelets provide best resolution of signals in data • We use tools developed at UW Wavelet IDR center • First step: Identify which filters isolate anomalies pb@cs.wisc.edu

  10. Analysis of “Normal” Traffic • Wavelets easily localize familiar daily/weekly signals pb@cs.wisc.edu

  11. Example Anomaly: Attacks • DoS: sharp increase in flows and/or packets in one direction • Linear splines seem to be a good filter to distinguish DoS attacks pb@cs.wisc.edu

  12. Characteristics of Flash Crowds • Sharp increase in packets/bytes/flows followed by slow return to normal behavior eg. Linux releases • Leading edge not significantly different from DoS signal so next step is to look within the spikes pb@cs.wisc.edu

  13. Characteristics of Network Anomalies • Typically a steep drop off in packets/bytes/flows followed a short time later by restoration pb@cs.wisc.edu

  14. Summary and Conclusion • Accurate network monitoring is essential for improving application performance and network management • The Wisconsin Advanced Internet Lab provides a unique environment for network monitoring • Wavelets are an effective means for identifying anomalous behavior in data gathered from IP flow and SNMP interface monitors • Details on distributed and coordinated monitoring and analysis available this spring pb@cs.wisc.edu

More Related