An Introduction of Botnet Detection – Part 1

An Introduction of Botnet Detection – Part 1 Guofei Gu, Wenke Lee (Georiga Tech)

Reference • Guofei Gu, Wenke Lee, et al. • BotHunter: Detecting Malware Infection through IDS-driven Dialog Correlation • USENIX Security 2007 • BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic • ACM NDSS 2008 • BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-independent Botnet Detection • USENIX Security 2008 • Moheeb Abu Rajab, et al. • A Multifaceted Approach to Understanding the Botnet Phenomenon • ACM IMC 2006 Speaker: Li-Ming Chen

Communication Attack Propagation What is Botnet? (1/2) • Bots: compromised hosts, “Zombies” • Botnets: networks of bots that are under the control of a human operator (botmaster) • (generally looks like) Worm + C&C channel • Command and Control Channel • Disseminate the botmasters’ commands to their bot armies (IRC, HTTP, … (can be encrypted)) (DoS, spamming, phishing site, …) Worm (vulnerabilities, file sharing, P2P, …) Speaker: Li-Ming Chen

What is Botnet? (2/2) • C&C Channel – Comm. protocols: • Most popular: IRC (Internet Relay Chat) • Open-source protocol, flexible • Others: HTTP, P2P… • Uses of Botnets • DDoS • Spam • ID/information theft • Phishing attacks • Distributing other malware Speaker: Li-Ming Chen

Lifecycle of a Typical Botnet Infection • Major roles: • Botnet (bots) • Victim • C&C Server • Botmaster authentication (optional) (borrow infection strategies from traditional malicious attacks) 6. Malicious activities (e.g., DDoS) Speaker: Li-Ming Chen

Why Botnet is hard to detect? • Botnet infection involves multiple steps • Only looking at one specific aspect likely to fail • However, predefined state transition models do not work well in botnet infection monitoring • Due to: • Rare to accurately detect all steps • Difficult to predict the order and time-window in which these events are recorded • Botnet can have very flexible design of C&C channels Speaker: Li-Ming Chen

Overview of the 3 Approaches BotMiner (Security’08) Speaker: Li-Ming Chen

Outline • What is Botnet? • BotHunter – Detecting Infection Lifecycle • BotSniffer – Detecting C&C Channel • BotMiner – Protocol- and Structure-independent Botnet Detection • My Comments Speaker: Li-Ming Chen

BotHunter (USENIX Security’07) • Snort-based sensor suite for botnet infection detection • Recognize the infection and coordination dialog that occurs during a successful bot infection • Observe the 2-way communication flows between internal assets and external entities • Identify data exchanges that match a state-based infection sequence model (by dialog correlation) Speaker: Li-Ming Chen

BotHunter System Architecture Recognize bi-directional warning signs of local infection Correlate this evidence against the defined dialog infection model dialog transitions Allows user to report bot infection profiles for global evaluation Speaker: Li-Ming Chen

SCADE & SLADE • SCADE (sCan Anomaly Detection) • Inbound & outbound scan detection (E1 & E5): • Based on protocol and Dst. port, monitor number of scans to or from local hosts • Assign weights to different ports and compute anomaly score for each local host • SLADE (payLoad Anomaly Detection) • Based on n-gram byte distribution anomaly detection • More robust to polymorphic blending attack Speaker: Li-Ming Chen

Bot Infection Dialog Model (1/2) • Design bot infection dialog model for assessing bi-directional flows across the network boundary • Roles: • A – attacker, V – victim, C – C&C server • 5 potential dialog transitions: • E1: external to internal inbound scan • E2: external to internal inbound exploit • E3: internal to external binary acquisition • E4: internal to external C&C communication • E5: internal to external outbound infection scanning Speaker: Li-Ming Chen

Bot Infection Dialog Model (2/2) Not strict ordering of events, but a typical infection dialog • (BotHunter) min. requirement • for bot declaration: • E2 AND E3-E5 • At least two distinct signs of • E3-E5 • Assign weights to different events • And then perform • correlation Speaker: Li-Ming Chen

Network Dialog Correlation Matrix Summarize ongoing dialog warnings for a specific local host Sensor alerts for each dialog warning • Each dialog might have 1 or 2 expiration intervals • (soft/hard prune timer) • When timer expires, compute dialog threshold score and • detect bot based on 2 conditions Speaker: Li-Ming Chen

When a dialog sequence is found and cross the threshold for bot declaration, BotHunter produces a bot profile Represents a full analysis of roles of the bot dialog Output: Bot Infection Profile (Example of a BotHunter profile) Speaker: Li-Ming Chen

BotSniffer (ACM NDSS’07) • Identify centralized botnet C&C channels in a monitored network • Including C&C servers and bots • Why focus on C&C channel? • C&C is essential to a botnet • Without C&C, bots are just discrete, unorganized infections • C&C detection is important • C&C channel is relatively stable and unlikely to change within botnets • Botmaster control bots via C&C channel (weakest point) Speaker: Li-Ming Chen

BotSniffer – the Approach • Observation: • Due to the pre-programmed activities related to C&C, • Bots within the same botnet will likely demonstrate spatial-temporal correlation and similarity • BotSniffer: • Focus on IRC & HTTP based C&C channels • Capture spatial-temporal correlation in network traffic • Utilize statistical algorithms to detect botnets • Has theoretical bounds on FP and FN rates Speaker: Li-Ming Chen

Centralized C&C Channels • Botmaster can control bots • via broadcast (real-time control) • Bots respond to the commands • in pre-programmed fashions • Relatively loose behaviors • (not real-time) • Sets the command in a file Speaker: Li-Ming Chen

Spatial-Temporal Correlation and Similarity • Regardless of the push and pull style • Invariants in botnet C&C channel: • 1. bots need to connect to C&C servers • (Virtually) long-lived session of C&C channel • 2. bots need to perform tasks and respond to the received commands (and in a similar fashion) • Message response (IRC-based reply) • Activity response (perform malicious tasks) Speaker: Li-Ming Chen

Response Crowd of Botnet Members Bots have much stronger (and more consistent) synchronization and correlation in their responses than normal users Speaker: Li-Ming Chen

BotSniffer System Architecture (data reduction) Port-independent, payload inspection (focus on IRC) Speaker: Li-Ming Chen

Correlation Engine • Group clients according to their Dst. IP and ports • Perform group analysis of spatial-temporal correlation and similarity based on two properties • Response Crowd Density Check • (Quantity  everybody acts!) • Response CrowdHomogeneityCheck • (Quality  everybody acts in the same way!) Speaker: Li-Ming Chen

Response Crowd Density Check • For each time window, check if there is a dense response crowd in a group • E.g., > 50% group members have message/activity behavior • Use TRW (threshold random walk) to compute the anomaly score and detect a sequence of crowds (H1: Botnet) Pr(Yi | H1) = θ1 Pr(Yi | H0) = θ2 (H0: not Botnet) Likelihood that botnet detected the i-th response crowd is dense ? Speaker: Li-Ming Chen

Response CrowdHomogeneityCheck • Check if most of the group members have very similar response • (currently only used for message response, IRC) • Also use TRW, but how to get Yi ? • Yi the i-th response crowd is homogeneous ? • Use a clustering technique to obtain the largest cluster of similar messages in the crowd • And calculate the ratio of the size of the cluster over the size of the crowd • Ratio > threshold  Yi = 1 Speaker: Li-Ming Chen

BotMiner (USENIX Security’08) • Why do we need BotMiner? • Botnets can change their • C&C content(encryption, etc.), • Protocols (IRC, HTTP, etc.), • Structures (P2P, etc.), • C&C servers, • Dialog models •  Bothunter, BotSniffer may be evaded (We need to consider more) Speaker: Li-Ming Chen

BotMiner – the Goal • Detect groups of compromised hosts within a monitored network that are part of a botnet • Not concern the way hosts get infected • The approach is • Independent of the protocol and structure used in C&C channel • Independent of the content of the C&C communication • Low FP and FN • Efficient Speaker: Li-Ming Chen

BotMiner – the Approach • Botnet is “a coordinated group of malware instances that are controlled viaC&C channels” •  monitor botnet in two planes: • C-plane (C&C communication traffic) • “who is talking to whom” • A-plane (malicious activity traffic) • “who is doing what” •  Find a coordinated group pattern in both kinds of activities Speaker: Li-Ming Chen

BotMiner System Architecture • Extract features from the raw logs • and perform clustering • Combine results and make final • decision Using different methods to analyze outbound traffic (based on Snort) log log Record flows, contact activities Speaker: Li-Ming Chen

C-Plane Clustering • 4 features: • temporal – fph, bps • spatial – ppf, bpp Further reduce traffic workload Data reduction, (Filter out irrelevant flows) Make clustering more efficient 2-step clustering, Coarse-grained clustering + fine-grained clustering (why?) Speaker: Li-Ming Chen

A-Plane Clustering • 2-layer clustering • based on activity type and features • (more straightforward) Speaker: Li-Ming Chen

Cross-Plane Clustering • Idea: crosscheck clusters in the two planes to find out intersections that reinforce evidence of a host being part of a botnet • 1. Botnet score s(h) for host h • 2. find similarity between bots (hi) and cluster Ai Aj .h (weight) 交集占聯集的比例 Speaker: Li-Ming Chen

Summary • Bothunter: • Vertical Correlation • Correlation on the behaviors of single host • Botsniffer: • Horizontal Correlation • Focus on centralized C&C botnets • Botminer: • Extension on Botsniffer • No limitations on the C&C types. Speaker: Li-Ming Chen

Botnet Detection – Part 2 • Focus on detailed approaches • Focus on evaluation methodologies and results • Possible evasions and solutions in Botnet detection • Discussion Speaker: Li-Ming Chen

My Comments • Divide and conquer • Understand the detailed attack behaviors • Try to detect attacks by correlating attack features • Attacks are anticipated to be more stealthy • Sophisticated, multiple stages... • Other evasion techniques •  complex detection approaches that make more assumptions about the attack might work well for that specific attack • But, not robust (easy to evade) Speaker: Li-Ming Chen

An Introduction of Botnet Detection – Part 1