BotGAD : Detecting Botnets by Capturing Group Activities in Network Traffic

1. BotGAD: Detecting Botnets by Capturing Group Activitiesin Network Traffic HyunsangChoi, Heejo Lee, and Hyogon Kim COMSWARE '09, Proceedings of the Fourth International ICST Conference on COMmunication System softWAre and middlewaRE Presenter: Yi Ning Chen

2. Outline • Introduction • Related Work • Group Activity of Botnet and Detection Scheme • Case Study: BOTGAD Using DNS • Conclusion

3. Introduction • A Botnet is a network of compromised machines controlled by an attacker to carry out online criminal activities including identity theft, e-mail spam, click fraud and DDoS attack. • A botnet'smaster can control the group remotely by command-and-control server (C&C server)

4. Difficulties of botnet detection • Botnet traffic is hard to detect because it is similar to normal traffic. What is worse, it may contain encrypted communication. • Botnets evolve quickly as more users fail to protect their computers, helping the attackers evade existing protection mechanisms. • Even botnet detections method can capture botnets which use the evasion techniques, most usually need huge amount of data which cannot be analyzed in real-time

5. Related Works • BotSniffer (2008) • BotSniffer has a similar concept with BotGADin respect of capturing the synchronized botnet communication. • Different from BotGAD, BotSniffer performs string matching to detect similar responses from botnets. • BotMiner (2008) • presents a botnet detection method which clusters botnet’s communication traffic and activity traffic.

6. Observation of Botnet • We find a common property of botnets: group activity. • Bots receive/send control traffic, download new codes, migrate the communication channel, and perform malicious behaviors.

7. Botnet Life Cycle Bots →DNS server, C&C server Bots →Target host Bots → C&C server

8. Group Activity of Botnet • Centralized botnets (HTTP and IRC) • P2P botnets • group activities can be observed during upgrading/ synchronizing

9. Two Cases of Group Activities • Suppose that we monitor incoming and outgoing traffic at a network gateway. Target Bot Bot Bot Bot Bot Target Bot Bot Bot Incoming group activity Outgoing group activity

10. Internal and External Group • ti: internal target • te: external target • An internal and an external group (Gi, Ge) which perform activity a to external/internal target within a time window wn, • Gi = {a, te, wn} • Ge= {a, ti, wn}

11. Characteristics of Botnet Group Activity

12. Similarity • Assume that a group is observed Gwithin wnand G’ within wn+1 • To measure the group uniformity, we compute a similarity between G and G’ • Kulczynski similarity • Cosine similarity • Jaccard similarity

13. BotGAD Detection Framework

14. Data Collection & Group Classifier If IP Addr 1 perform the group activity within w1

15. Similarity Estimation Methods

16. Estimate Group Properties – Average Similarity • Some botnet groups can be seen in wi, not in wn+1 due to the relatively small value of w choice. • Therefore, we delete deficientcolumn vectors which satisfy (m is the number of hosts in the group.) • Average similarity value within a given monitoring time t (t=nw)

17. Estimate Group Properties – Periodicity & Intensity • Periodicity • If the periodicity P is equal to zero, the group entries occurred periodically at each time window • Intensity • If the intensity is equal to one, the group entries appear intensively. • A lot of groups founded in normal communication patterns, do not appear intensively

18. Identify Botnet • With the combination of average similarity, periodicity and intensity, BotGAD decides whether a groups is a botnet or not. • If average similarity > λD, the group is considered suspicious. • Delete false positives which have intensity < λI • Among remainder groups, if periodicity < λP, we judge the groups are periodic bots

19. CASE STUDY: BOTGAD USING DNS

20. DNS Used in Botnets(1/2) • Rally • If a host infection succeeds, the host send DNS query to know the name of a C&C server. • Update • Botnetsusually update their codes with the latest one by downloading it from their web repository. the botnets find the repository using DNS. • Synchronization • Some botnets synchronize the system time of infected machines with the Network Time Protocol (NTP) using time server DNS (e.g., Storm worm botnet [16]).

21. DNS Used in Botnets(2/2) • Cloning and Reconnection • Bots frequently do cloning and reconnecting to be undetectable. At the moment, bots find their new/old channel servers using DNS. • Migration • Botnetsmigrate C&C servers using DNS. • Attack • Spamming, DDoS attack and click fraud attacks may use DNS to find victims.

22. Experiments • Collect DNS traces tapped from the gateway router of /16 campus network. • Experiment #1 on 2008/5/19 • 6.28GB of DNS traffic and 19.52 million DNS queries • Observed average 640,000 domain groups, but only 8% of the groups (51,200) have more than 3 hosts. • Decide group size threshold, λS to be 3 • Experiment #2 on 2008/12/24 • 1.48GB of DNS traffic and 4.6 million DNS queries • DNS queries are decreased remarkably because the NAC (Network Access Control)

23. Measured 3 Different Similarities • Experiment#1 • w: 10 minute, t: 1 hour

24. Comparison of Experiment Results • The comparison infer that the NAC solution affects positively to BotGAD

25. Dealing with False Positives • After applied λI , there were still some false positives. Most are update related domains, which can be removed using white list.

26. Evadability of BotGAD • If bots intentionally generate fake DNS queries using source address spoofing, the fake queries can poison BotGAD. • We can check follow-up TCP connections of DNS queries to delete the fake queries.

27. Conclusion • We define an inherent property of botnets, called group activity. • We develop metric model to measure the property and detection mechanism which can detect botnetsfrom large scale networks in real-time. • We implemented BotGAD using DNS traffic as a case study and the effectiveness of the implemented system by the experiments on real-life campus network trace.