Analyzing and Exploiting Network Behaviors of Malware

1. Analyzing and Exploiting Network Behaviors of Malware Jose Andre Morales Areej Al-Bataineh ShouhuaiXu Ravi Sandhu SecureComm Singapore, 2010 World-Leading Research with Real-World Impact!

2. Introduction Do malicious and benign processes behave differently from a networking perspective? Can we exploit these differences to identify malware, especially zero-day attacks? Analyzed 1000 malware samples, with 31 not detected by Virustotal.com 01 April 2010 and 123 benign samples Focus on DNS, NetBIOS, TCP, UDP, ICMP World-Leading Research with Real-World Impact!

3. Introduction - 2 • Log file analysis tallied various network event occurrence amounts • Along with traffic observations we identified behavior occurring mostly in malware • Defined 7 behaviors dealing with specific observed anomalies in network traffic • Some behaviors combine network events to form an anomaly • These behaviors used to differentiate between malicious and benign processes • Clustering and classification World-Leading Research with Real-World Impact!

4. Contributions • Identification of network behaviors occurring mostly in malware usable in behavior based malware detection. • Discovery of novel malicious uses of network services by malware. • Evaluating the effectiveness of observed network behaviors in identifying malware and benign processes with clustering and classification. World-Leading Research with Real-World Impact!

5. 7 Behaviors • B1: Process performs a NetBIOS name request on a domain name that is not part of a DNS or rDNS query • B2: Failed connection attempt to an IP address obtained from a successful DNS query • B3: Failed connection attempt to the input IP address of a successful rDNS query • B4: Connection attempt to the input IP address of a failed rDNS query World-Leading Research with Real-World Impact!

6. 7 Behaviors • B5: ICMP only activity, ICMP echo requests for a specific non-local network IP address with no reply or a returned error message. • B6: TCP/ICMP activity, TCP connection attempts to non-local IP addresses that received a successful reply to their ICMP echo requests • B7: Network activity that is rarely occurring or implemented in an anomalous manner World-Leading Research with Real-World Impact!

7. Behavior B1 • Process performs a NetBIOS name request on a domain name that is not part of a DNS or rDNS query • Table shows B1 occurring only in malware, benign NetBIOS used domain names previously used in a DNS query. • Several domains in B1 known malicious by Malwareurl.com but others were not World-Leading Research with Real-World Impact!

8. Behaviors B2, B3 & B4 • DNS often used to acquire IP addresses • Only B2 occurred, many malware DNS domain names and cannot connect with returned IP, either offline or shutdown, or newly registered and inactive • B3, B4 no occurrence, possible less favored by malware authors World-Leading Research with Real-World Impact!

9. Behaviors B5 & B6 • ICMP used by malware (like PING) to acquire active IP addresses, these IPs not part of previous DNS, rDNS or NetBIOS  suspicious behavior. B6 dominant in malware • B5 almost same in both, very similar to DNS behavior with no request reply World-Leading Research with Real-World Impact!

10. Behavior B7 • Considered suspicious but not necessarily malicious, behaviors were rarely occurring or implemented in non-conventional manner • TCP connection attemps most prevalent, IP not acquired via DNS, NetBios or ICMP, possibly hardwired or dynamically generated World-Leading Research with Real-World Impact!

11. Behavior Evaluation • 1000 malware samples from CWSandbox 27 October 2009 upload, diverse set, still active durng testing. • 31 samples from 31 March 2010 upload not detected by Virustotal.com (MD5 search) 1 April 2010 • 41 benign samples executed 3 times each = 123 total benign samples – FTP, RSS, socnet, P2P, AV, net tools • Individual samples run for 10 minutes in VMWare (XP SP2) using Windows network monitor, proprietary netwok layer filters • Results revealed behaviors differentiate malicious from benignincluding 31 unidentified malware World-Leading Research with Real-World Impact!

12. Clustering & Classification - 01 Weka data mining software Clustering used complete malware and benign data set Classification training set used 1st 700 malware samples and 40 benign, testing used the remaining samples 31 unknown samples not part of training set World-Leading Research with Real-World Impact!

13. Clustering & Classification - 02 World-Leading Research with Real-World Impact!

14. Clustering Results • If majority of cluster was malware then benign samples assumed FP, If majority of cluster was benign then malware samples assumed FN • Xmeans perfect, DBScan & EM encouraging • All 31 unknown malware correctly identified • FP video streamers known to be unreliable networks • EM FN mostly malware downloaders World-Leading Research with Real-World Impact!

15. Classification Results • FN and FP very low, 2 malware flagged as FN by all 4, only 2 video streams flagged as FP • 29 unknown malware correctly identified by all 4 World-Leading Research with Real-World Impact!

16. Discussion B1, B2 & B7 most dominant behaviors B1,B5 & B6 considered novel behaviors used by malware to find active remote hosts Classification & clustering produced excellent results with minimal FN & FP 31 malware not identified by virustotal.com on 1 April 2010 were correctly detected with minimal exceptions World-Leading Research with Real-World Impact!

17. Conclusions Network behaviors can be exploited to differentiate between malicious and benign Discovered 3 novel network behaviors Our approach can be combined with other perspectives to enrich detection accuracy The behaviors detected a diverse set of malware inlcuding 31 unknown samples with minimal FP and FN World-Leading Research with Real-World Impact!