An Introduction of Botnet Detection – Part 2

1. An Introduction of Botnet Detection – Part 2 Guofei Gu, Wenke Lee (Georiga Tech)

2. Reference • Guofei Gu, Wenke Lee, et al. • BotHunter: Detecting Malware Infection through IDS-driven Dialog Correlation • USENIX Security 2007 • BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic • ACM NDSS 2008 • BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-independent Botnet Detection • USENIX Security 2008 • Moheeb Abu Rajab, et al. • A Multifaceted Approach to Understanding the Botnet Phenomenon • ACM IMC 2006 Speaker: Li-Ming Chen

3. Lifecycle of a Typical Botnet Infection • Why Botnet is hard to detect? • involving multiple steps • flexible design of C&C • channels authentication (optional) 6. Malicious activities (e.g., DDoS) (borrow infection strategies from traditional malicious attacks) Speaker: Li-Ming Chen

4. C&C (Command and Control) Channels Centralized C&C channel Message Response Crowd Activity Response Crowd P2P C&C channel Speaker: Li-Ming Chen

5. Comparison of the 3 Approaches Speaker: Li-Ming Chen

6. Predefined Lifecycle BotHunter Signs match the predefined evidences (dialog transitions) Utilize Snort to detect sign of local infection • A Bot could be: • E2 AND E3-E5 • At least two distinct • signs of E3-E5 Speaker: Li-Ming Chen

7. BotHunter (cont’d) • Anomaly-based payload exploit detection • Learn normal profile (using 2-gram PAYL) • Check deviation distance of a test payload from the normal profile • Current bots are multi-vector • Design two modules (inbound/outbound) • for scan detection • Assign high weight to ports often used • by malware (predefined) • Observe outbound scan rate, outbound • connection failure rate, and address • dispersion • Use bot-specific heuristics to build signatures (rules) Speaker: Li-Ming Chen

8. BotHunter:Evaluation Results (1/2) • Experiments in a virtual network • To test FN rate (by examining 10 different bots) # involving the victim # of generated dialog warnings Speaker: Li-Ming Chen

9. BotHunter:Evaluation Results (2/2) • Honeynet-based experiments • Use SRI honeynet to capture real-world bot infection • Use BotHunter to analysis these traces • 95.1% TP rate (1920/2019 in 3 weeks) • FN is due to: • Infection failure, honeynet setup and policy failure, data corruption failure. • Experiments in a campus network • 98 profiles were generated in 4 months (no FP) • Experiments in SRI laboratory network • Generate 1 bot profile and it is FP (a 1.6 GB multifile FTP transfer matchs “E2 & E3”) Speaker: Li-Ming Chen

10. BotHunter:Pros and Cons • Pros: • Real-time detection of bot infections • Evidence trail gathering for investigation of putative infections • Cons: • Use heuristic (2 conditions) to decide a bot infection • Less flexible Speaker: Li-Ming Chen

11. BotSniffer • Response crowd: • Density check • Homogeneity check (data reduction) Port-independent, payload inspection Speaker: Li-Ming Chen

12. BotSniffer:Evaluation Methodology • Use normal traffic traces to test the FP rate and use botnet traces (mix normal traffic) to test the detection performance • Normal traces: • Capture 8 IRC traces (port 6667) and 5 complete traces from campus network • Botnet traces: • Collect 3 real-world IRC-based botnet traces • Generate 3 botnet traffic by modifying source codes of 3 common botnets • Implement 2 http-based botnet Speaker: Li-Ming Chen

13. BotSniffer:Evaluation Results (1/2) All FP are generated due to single client incoming message response analysis. (Apply both activity response and message response group analysis) Speaker: Li-Ming Chen

14. BotSniffer:Evaluation Results (2/2) honeynet IRC logs (both message and activity) (periodically connect to server) (random delay) (the randomization of connection periods did not cause a problem, because there were still several clients performing activity responses at the time window) Speaker: Li-Ming Chen

15. BotSniffer:Pros and Cons • Pros • Successfully detect all botnets (low FP rate) • Efficient alert reduction • More robust than other botnet detection system • Cons • Focus on centralized C&C communication • Configure time window for group analysis • Possible evasions (e.g., misusing whitelist, encryption, protocol matcher, long response delay, obfuscation) Speaker: Li-Ming Chen

16. BotMiner (similar to BotSniffer) (more straightforward) log • Combine results and • make final decision log (more complex) Focus on flow statistics, not message response! Speaker: Li-Ming Chen

17. BotMiner: Evaluation Methodology • (same) use normal traffic traces to test the FP rate and use botnet traces (mix normal traffic) to test the detection performance • Normal traces: • Capture 10 days traffic record at the campus network • Botnet traces: • 4 IRC, 2 HTTP and 2 P2P botnets • 2 IRC and 2 HTTP are also used for BotSniffer • P2P: 2 real-world traces (Nugache and Storm) TCP, encrypted UDP Speaker: Li-Ming Chen

18. BotMiner: Evaluation Results (1/3) (C-plan data reduction) Most useful, Only record internal to external flows. Remove helf-open TCP flows Whitelist Speaker: Li-Ming Chen

19. BotMiner: Evaluation Results (2/3) • 4 features: • temporal – fph, bps • spatial – ppf, bpp Further cluster by separating each feature as a vector of 13 elements according to their distribution Cluster by using the mean and variance of the features Most FP clusters contain only 2 hosts Ignore clusters only contain 1 host Speaker: Li-Ming Chen

20. BotMiner: Evaluation Results (3/3) FN Speaker: Li-Ming Chen

21. BotMiner:Pros and Cons • Pros: • Anomaly-based botnet detection system (independent of the protocol and structure used by botnets) • Low FN and FP rate • Cons: • Stealthy: botmaster can commond the bots to perform extremely delayed task (evade cross clustering) Speaker: Li-Ming Chen

22. Summary • Bothunter: • Vertical Correlation • Correlation on the behaviors of single host • Botsniffer: • Horizontal Correlation • Focus on centralized C&C botnets • Botminer: • Extension on Botsniffer • No limitations on the C&C types. Speaker: Li-Ming Chen