Early Detection of Outgoing Spammers in Large-Scale Service Provider Networks

1. Early Detection of Outgoing Spammers in Large-Scale Service Provider Networks Yehonatan Cohen Daniel Gordon Danny Hendler Ben-Gurion University Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

2. Talk outline • Preliminaries • ErDOS: An Early Detection Scheme for Outgoing Spam • Evaluation • Conclusions and Future Work Danny Hendler and Philipp Woelfel, PODC 2009

3. Preliminaries • Spam Unsolicited mail, typically sent in large quantities • Hazards • Malware distribution • Phishing • Resource consumption • Poor user experience • Detection may be attempted when • Mail is sent (outgoing spam detection) • Mail is received (incoming spam detection) Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

4. Outgoing spam detection Spam can be blocked before leaving the Email Service Provider (ESP) Advantages Reduces load on ESP infrastructure Prevents damage to ESP reputation Detection may be based on hosted accounts' activity Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

5. Outgoing spam filtering techniques Contents-based filtering: Learn & identify messages' textual patterns typical of spam messages May be tricked by manipulating spam content Image-based Random string insertion (hash busters) Non-negligible false negative rate Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

6. Outgoing spam filtering techniques (cont'd) • Inter-account communication patterns analysis: • Models accounts' behaviour • Based on inter-account social interactions • Typically utilizes machine-learning techniques • May leverage ESP account identification Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

7. Our goals • Devise an effective detector of outgoing spammers for large ESPs (the ErDOS detector) • Emphasis on early detection • Detects spammers before the contents-based filter • Short training periods • Highly adaptive to changing spamming patterns Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

8. Most relevant related work Lam & Yeung, CEAS 2007 Introduce “social-network”-based outgoing spam detection Use the k-NN classifier Relatively small dataset (ENRON) Labeling based on simulated spammer accounts Tseng & Chen, CSE 2009 Uses same set of features Uses SVM classifier Larger, non-ESP dataset (University email server) Incremental model update Labeling based on pure accounts Account identification based on “from” header field Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

9. Comparison with data-sets of previous work • Collected by a very large ESP • Consists of incoming and outgoing log files • 4 days of bi-directional data + 22 days of outgoing traffic only • Both incoming and outgoing messages are labeled as spam/ham by a content-based detector Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

10. Comparison with data-sets of previous work • Collected by a very large ESP • Consists of incoming and outgoing log files • 4 days of bi-directional data + 22 days of outgoing traffic only • Both incoming and outgoing messages are labeled as spam/ham by a content-based detector Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

11. Talk outline • Preliminaries • ErDOS: An Early Detection Scheme for Outgoing Spam • Computation Flow • Features • Evaluation • Conclusions and Future Work Danny Hendler and Philipp Woelfel, PODC 2009

12. The ErDOS detector: computation flow Pre-processing Feature values computed Compute account feature values based on a single day of email logs Determine accounts' classification Undersampling: extract all spammers and equal number of legitimate accounts as training set Classified data set Training set Construct suspect accounts list of configurable size Assign account scores using classification model Build rotation forest model Scored accounts Classification model Remainder of accounts not in training set Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

13. Talk outline • Preliminaries • ErDOS: An Early Detection Scheme for Outgoing Spam • Computation Flow • Features • Evaluation • Conclusions and Future Work Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

14. ErDOS features: IOR • An account’s IOR = #incoming/#outgoing mails Legitimate users Spammers Sent messages seldom replied • Maintain social interactions • Often belong to mailing lists Low IOR characteristic of spammers Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

15. ErDOS features: IOR (cont'd) Danny Hendler and Philipp Woelfel, PODC 2009

16. Communication Reciprocity (CR) Fraction of recipients who responded to an account's emails Defined by Gomes et al. IOR is superior for short training periods ErDOS features: IOR versus CR Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

17. IEBC (Internal/External Behaviour Consistency) An account can send/receive emails to/from Internal addresses (accounts hosted by ESP) External addresses Legitimate accounts show correlation between internal and external IOR, spammers less so ErDOS features: IEBC Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

18. ErDOS features: #outgoing messages Number of outgoing messages Spamming accounts send more emails than legitimate Insufficient for detecting low-volume spammers Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

19. A large fraction of spammers' incoming mail is spam! Legitimate accounts seldom send emails to spamming accounts Dictionary attacks may cause spammers to spam each other Analyse senders' characteristics ErDOS: Sender Accounts' Characteristics Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

20. Talk outline • Preliminaries • ErDOS: An Early Detection Scheme for Outgoing Spam • Evaluation • Conclusions and Future Work Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

21. Accuracy for Single-Day training Evaluate Accuracy attained for single day logs Email accounts are classified based on the tags of the contents-base detector True Positive (TP) and False Positive (FP) values are averaged over available 4 days of bidirectional data Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

22. Early detection evaluation Spamming accounts detected before thecontents-based detector Suspected by detector, send messages tagged as spam only on later days Evaluation uses all 26 days of data Early detection quality criteria: e-Precision: fraction of early detected accounts out of suspects list. Enrichment Factor (EF): ratio between detector'se-Precision and that of a random accounts list. Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

23. Early detection Early detection results, averaged over 4 days: Prior art’s early detections results compared to ErDOS: Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

24. Early detection (cont’d) e-Precision for varying suspects list lengths: Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

25. Talk outline • Preliminaries • ErDOS: An Early Detection Scheme for Outgoing Spam • Evaluation • Conclusions and Future Work Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013

26. Conclusions and Future Work Conclusions The case of outgoing spam detection for ESPs has its unique nature Contents-based filtering is not enough Early detection of spamming accounts can be achieve by a combination of contents-based filter and network level-based detector Future Work Enhancement of ErDOS’s early detection performance by additional features A low-volume spammers expert detector, based on ErDOS’s computation flow and features Yehonatan Cohen, Daniel Gordon and Danny Hendler, DIMVA 2013