1 / 36

Fighting Spam

Randy Appleton Northern Michigan University rappleto@nmu.edu. Fighting Spam. What is Spam. Probably, it’s “unsolicited and unwanted commercial email sent in bulk”. Sometimes It’s Not Spam. You did sign up for it. You accidentally signed up for it. You still don’t want it.

alexander-morris
Download Presentation

Fighting Spam

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Randy Appleton Northern Michigan University rappleto@nmu.edu Fighting Spam

  2. What is Spam • Probably, it’s “unsolicited and unwanted commercial email sent in bulk”.

  3. Sometimes It’s Not Spam • You did sign up for it. • You accidentally signed up for it. • You still don’t want it.

  4. How Is It Delivered? • Anyone can fake email. • 80% of all spam came from bot-nets • We helped  • Open relays are mostly gone. • You can hire this done for you (see Google).

  5. How Much Spam Is There? • In absolute numbers • 1978 - An e-mail spam is sent to 600 addresses. • 1994 - First large-scale spam sent to 6000 bulletin boards, reaching millions of people. • 2005 - (June) 30 billion per day • 2006 - (June) 55 billion per day

  6. How Much Spam Is There #2 • As a percentage of the total volume of e-mail • MAAWG estimates that 80-85% of incoming mail is "abusive email", as of the last quarter of 2005. The sample size for the MAAWG's study was over 100 million mailboxes. • More is coming!!!

  7. Why They Spam • Money • Political causes. • Money • It’s fun • Money • Money

  8. Sell You Something • It’s just mass electronic marketing • They give you a web site, you click over and buy the product. • Email might even be targeted.  • weight loss.html

  9. Does Selling By Email Work? • Kodak settled a CAN SPAM suit with the FTC. Their Ofoto unit sent two million commercial messages that didn't comply with the CAN SPAM act. They didn't include a notice that it was an ad, opt-out info, and Kodak's postal address. They paid the FTC $26,000, the revenue they got.

  10. Pure Fraud “There is a sucker born every minute.” • Send email to lots of people. • Wait for sucker to respond. • Convince them to give you money. • Nigerian bank fraud

  11. Identity Theft • Send an email message. • Direct them with a bad URL. • Capture their info. • Reject login and send them to the right site. • Microsoft says to manually check every link.

  12. Identity Theft #2 • An Example • Who Did It.

  13. Stock Manipulation • Pick a small cap stock • Buy some. • Send spam telling people about the stock. • Sell when price rises. • stock-spam.txt • spam-stock.jpg • New York Times

  14. Yes, Spam Works • 5% response rate from sexual material. • 0.02% response rate for drugs. • 0.0075% response rate for Rolex Watches.

  15. Avoiding Spam • Don’t let them get your email address. • Don’t use AOL, etc. • Don’t put address on web page. • Don’t use mailing lists. • Throw away email addresses. • Mailinator, spamgourmet, sneakermail • Annoying …. but possible.

  16. List Removal • For a reputable company, you can always click “remove me from the list”. • A disreputable company will merely take that to be confirmation you’re reading the email. • It’s a calculated gamble.

  17. Auto Detecting Spam • Blacklist • Whitelist • Bayesian Analysis • Other Analysis • These are all things your email server does for you.

  18. Blacklist • A list of web sites from which you don’t take mail. • Automatically interfaced to your email server. • Spamhaus Block List • Zelots • Many choices.

  19. Defeating Blacklists • The spammers can switch ISPs. • The spammers can use a botnet.

  20. Whitelist • There is no global whitelist; you make your own. • Your own contact group is a good start. • Add your institution. • Add people to whom you have sent mail. • Semiautomatic at best. 

  21. Bayesian Analysis • Make two piles of mail: spam and ham. • Find words or phrases that can be used to identify mail. • Check all incoming mail for those phrases. • Normally you get a starter database that can be customized.

  22. Example Bayesian Analysis • My friends don’t email me about Viagra. • They do email me about Linux. • The phrase “stupid freshmen” appears in email to me. • The phrase “hot freshman” does not. • Result is a score.

  23. Defeating Bayesian Analysis • Intentionally misspell words • Humans will still understand Vi agra. • But this is an even more potent of spam-hood. (unless you have unique misspelling, and you don't). • Use pictures instead of words. • But that indicates spam-hood too.

  24. Defeating Bayesian Analysis • Add happy words • Add poetry • move closer playoff berth Yankees straight East CNBC Stocks Source: HP got Fiorinas phone records Energy price drop makes Feds job easier farmers Oil continues slide gas Down Oprahs America live Clay Aiken releases third album turn Crocodile AND NASA spotting debris Probe spots around Saturn Distorted solar system Schools crack amp FITNESS More potent Test Pattern: Multilink Monday World poor real headlines Watch netcast man who believes planning make next nuclear

  25. Other Analysis • Look for a feature that is more likely in spam than ham. • Give that a score. • If the score is high enough, reject. • Can (and should) be combined with other techniques.

  26. SpamAssasin Rules • http://spamassassin.apache.org/tests_3_1_x.html.

  27. Other Analysis Example • 1.3 HELO_DYNAMIC_HEXIP Relay HELO'd using suspicious hostname (Hex IP) • 0.8 EXTRA_MPART_TYPE Header has extraneous Content-type:...type= entry • 2.0 DATE_IN_FUTURE_03_06 Date: is 3 to 6 hours after Received: date • 0.9 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of words • 0.0 HTML_MESSAGE BODY: HTML included in message • 2.0 RCVD_IN_SORBS_DUL RBL: SORBS: sent directly from dynamic IP address

  28. Other Analysis Example #2 • 0.9 FORGED_YAHOO_RCVD 'From' yahoo.com does not match 'Received' headers • 0.8 HTML_IMAGE_ONLY_32 BODY: HTML: images with 2800-3200 bytes of words • 2.2 REPTO_QUOTE_YAHOO Yahoo! doesn't do quoting like this • 0.2 MIME_BOUND_NEXTPART Spam tool pattern in MIME boundary

  29. Other Fun Ideas • Return “temporary error” code on first time senders. • Return “temporary error” code if you think the last email was spam. • Ring them back. • Challange-Response.

  30. Current Best Technique • If on whitelist, admit mail. • Check connection address via blacklist(s). • Check for spammy features. • Use Bayesian Analysis • Produce a combined score. • Sort on combined score.

  31. Success Rate • You can expect 80%-95% of spam to be detected. • You can expect nearly 0% of real mail to be misclassified. • Adding anti-virus detection is easy. • Adding porn detection is not reasonably possible.

  32. Fighting Back • Don’t. • The nasty email goes to an innocent. • Or it confirms you exist. • Or it bounces back to you.

  33. Using • Gmail filters. • Gmail allows pop downloads. • You can even forward the mail to Gmail to keep your old account name.

  34. Using JunkEmailFilter.com • Small business, $100 per year. • Large business, ¼ cent per email. • Progressive charities are free. • Lots of other choices.

  35. Buy a Box • Barracuda Networks will see you a box. • It will filter spam, viruses, and more. • Costs $2000. • Very low administration once set up.

  36. Summary • Keep email address private. • Use spam detecting software. • Use spam detecting software. • Or use Google. • Suffer.

More Related