1 / 23

URCA: Pulling out Anomalies by their Root Causes

URCA: Pulling out Anomalies by their Root Causes. Fernando Silveira and Christophe Diot. URCA: Pulling out Anomalies by their Root Causes. Presenter: Fernando Silveira UPMC and Technicolor. Joint work with Christophe Diot. Presented at INFOCOM 2010 – San Diego, USA.

jatin
Download Presentation

URCA: Pulling out Anomalies by their Root Causes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. URCA: Pulling out Anomalies by theirRoot Causes Fernando Silveira and Christophe Diot

  2. URCA: Pulling out Anomalies by theirRoot Causes • Presenter: Fernando Silveira • UPMC and Technicolor Joint work with Christophe Diot Presented at INFOCOM 2010 – San Diego, USA

  3. Traffic Anomaly Detection Alarm Anomaly Detector Traffic Data Anomaly Anomaloustraffic Packet counts Time

  4. Root Cause Analysis of Traffic Anomalies • Obtaining information about an anomaly’s cause. • Automating root cause analysis is important… • Manual analysis is tedious and error prone • Study from Arbor Networks with 67 ISPs • Average ISP observes ~ 19 anomalies/day • … but it is also a hard problem. • Most detectors do not provide any information beyond an alarm

  5. Related Work • Anomaly detection methods with properties that facilitate root cause analysis tasks • Anomaly classification • Lakhina et al. - SIGCOMM’05 • Based on clustering entropy residuals • Limited to anomalies found in entropy • Anomalous flow identification • Schweller et al. - IMC’04, Li et al. - IMC’06 • Based on reversible sketches • Complexity of choosing and computing sketches • Limited to anomalies found in sketches

  6. Our Contribution • URCA (Unsupervised Root Cause Analysis) • a tool that finds an anomaly’s root cause • can be used with different anomaly detectors • It provides accurate and fast results: • anomalies are analyzed as fast as they are detected (1-5 minutes)

  7. Outline Algorithms for URCA Performance Evaluation

  8. Our Approach URCA has two steps: • anomalous flow identification • root cause classification Our methods rely on flow features

  9. Step 1: Anomalous Flow Identification Alarm Filter Anomaly Detector Traffic Data Destination Port

  10. Flow Identification - Example Destination AS (3 values) Output Interface (2 values) AS 2108 Anomaly Packet counts Candidate flows Anomalous flows AS 3354 eth0 AS 1277 Normal flows Normal flows eth1 Time

  11. Visualizing Root Cause Flows Network scan Routing change

  12. Step 2: Root Cause Classification a a a a b b b ? c c c • We compute metrics from each anomaly • number of source IP’s, ASN’s, flow sizes, packet sizes, etc. • Hierarchical Clustering • known anomalies + 1 unknown • Bootstrapping labels • helped by visualization

  13. Outline Algorithms for URCA Performance Evaluation

  14. Experimental Methodology Traces from links in GEANT2 Anomalies obtained with the ASTUTE anomaly detector

  15. Identification Accuracy - Trace A

  16. Identification Accuracy - Traces B-F * 90-percentile averaged across traces

  17. Classification Accuracy - Trace A 80% Correct 5% require visualization 15% Misclass. 5% first occurrences of an event type + 10% routing changes mistaken for link failures 15% Misclassified =

  18. Wrapping Up • What you’ll find in the paper: • Algorithms for both identification and classification • Experimental evaluation with 6 traces • URCA can be applied to other anomaly detectors • Ongoing and Future Work: • URCA with an EWMA-based detector • Using other sources of data (e.g., routing data)

  19. The End • Special thanks to: • DANTE / GEANT2 - http://www.geant2.net/ • Ricardo Oliveira @ UCLA - http://irl.cs.ucla.edu/~rveloso/ • More information at: • http://www.thlab.net/~fernando/papers/urca.pdf • http://www.thlab.net/~fernando/papers/astute.pdf

  20. Backup Slides

  21. Classification results for ASTUTE

  22. Classifying the Unknown ASTUTE Anomalies

  23. Results with EWMA

More Related