1 / 7

EbAT: Online Methods for Detecting Utility Cloud Anomalies

EbAT: Online Methods for Detecting Utility Cloud Anomalies. Chengwei Wang CERCS, Georgia Institute of Technology Thesis Advisor: Prof. Karsten Schwan. Outline. Problem Description - Challenges of anomaly detection in utility cloud - State of the art - Positioning our solution

jolie
Download Presentation

EbAT: Online Methods for Detecting Utility Cloud Anomalies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EbAT: Online Methods for Detecting Utility CloudAnomalies Chengwei Wang CERCS, Georgia Institute of Technology Thesis Advisor: Prof. Karsten Schwan

  2. Outline • Problem Description - Challenges of anomaly detection in utility cloud - State of the art - Positioning our solution • Solution Description • Experiment Evaluation and Discussion - Experiment I: RUBiS Benchmark - Discussion: Hadoop Experiment

  3. Challenge I: Large Scale in Hardware 8’192’000 cores / 81’920’000 VMs on the Infrastructure. Billions of Metrics. Anomalies are buried like needles in a haystack!

  4. Horizontal Crossing Map-Reduce Social- Networking C2C/ B2B/ B2C Video- Sharing APP Level Vertical Crossing Guest OS Guest OS Guest OS Guest OS … … … OS Level Hypervisor Hypervisor … VM Level Platform Platform … Platform Level Challenge II: Large Scale in Software ‘Cloud’ Characteristics: Anomaly Detection Challenges: 1. Millions of Services 1. Scalability 2. Mixture of Heterogeneous Apps 2. Lack of Pre-Knowledge 3. Multi Levels of Software Stacks 3. Vertical/Horizontal Correlating 4. VM Creations/Migrations/Termination 4. Dynamism of Workload Patterns

  5. Problem Definition • Aggregation Problem 1. Scalability: reduce the data volume in communication and analysis 2. Retain valuable information for anomaly detection and identification. 3. ‘horizontal crossing' and 'vertical crossing‘: metrics in different levels/components are collectively considered • Detection Problem 1. Online designating when the utility cloud is experiencing anomalies. 2. High Detection Rate and Low False Alarm Rate 3. Unsupervised method with minimal pre-knowledge about normal or abnormal behaviors. • Zoom-In Problem Localizing anomalies so as to narrow the search scopes for further diagnosing the causes of those anomalies.

  6. State of the Art Issues • Industrial World: Threshold-Based Approaches 1. Incremental False Alarm Rate: the false alarm rate grows with increase in the number of monitoring metrics 2. Detection after the Fact 3. Scalability Issues: online monitoring huge volume of metrics • Academia: Statistical Approaches 1. Sophisticated and promising approaches for problem diagnosis. 2. None claims to be scalable. 3. Most of them use heavy-weight machine learning algorithms 4. Most of them requires pre-knowledge. Details in Backup Slices

  7. Outline • Problem Description - Challenges of anomaly detection in utility cloud - State of the art - Positioning our solution • Solution Description • Experiment Evaluation and Discussion - Experiment I: RUBiS Benchmark - Discussion: Hadoop Experiment

More Related