220 likes | 314 Views
Mail-Filters Technical Presentation. How it works, Why it’s Better. Mail-Filter Technology Overview. Why Mail-Filters Bullet Signature Creation Star Engine Process Overview Implementation Options SDK Contents Getting Started The API Commands Testing Options OEM Implementation Examples
E N D
Mail-Filters Technical Presentation How it works, Why it’s Better
Mail-Filter Technology Overview • Why Mail-Filters • Bullet Signature Creation • Star Engine Process Overview • Implementation Options • SDK Contents • Getting Started • The API Commands • Testing Options • OEM Implementation Examples • FAQs
Why Mail-Filters • It’s Fast – 100s of messages per second (or higher) • It’s Accurate – over 95% of spam caught, less than 1 in 1,000,000 false positive rate • Many implementation options – the right solution for any environment • It’s Proprietary – it’s not fooled by spammer tricks - gives time to market and competitive differentiation • It catches Foreign Language Spam – in over 30 languages – a worldwide solution • Easy Implementation – usually less than a day • Full Support – Integration, technical support and training, marketing materials, sales training and lead generation
How Mail-Filters Works 1.Spam Collectionoccurs from many sources 5.TuningUsers and Administrators provide feedback to help identify spam and those that send them. 2.Human EditorsCraft Bullet Signatures 3.Bullet SignaturesAre Updated Every 1-15 Minutes 4.Mail-Filters Technology Integrated into OEM Solutions- Catches Spam, without False Positives
Mail-Filters’ Process Overview To Capture Spam & Create Bullet Signatures Mail-Filters Data Centers Bullet Signature Updater Customer Bullet Signatures Customer submissions Traffic and Connection Heuristics Mail-Filters Technology on Customer Device www Auto-Nominate Process Phish Trolling Spam DB Culling Engine Bullet Signature Updates International Spam Harvester Scam Sensors www Traffic Analysis Phish Traps Language Assignment Prioritization Process Pre-Qualified & Auto-Nominated Partner Collections Reputation Analysis Data Quality Manager Aristotle (Signature Auto-Suggest) Spam Pre-Qualification Partner Pre-Qualification Expert Bullet Signature Creation Quality Check Spammer Profile Creation Translation Tools Message Profile Creation Human Editors Traffic Profiles
Star Engine Process Overview Mail-Filters Data Centers STAR Engine Server Known Good Mail Message Normalizer OEM Software SnowFlake Buster Bullet Signature Updater Language Analyzer Malformed Message Processor Is Message Spam? STAR Engine Management Module Star Engine Interface Message Analysis Bullet Signatures Yes / No Traffic Analysis Reputation Analysis Spammer Profile Check False Positive Rationalizer
Implementation Options • Enterprise • Most typical implementation – highest performance – uses more resources • Desktop • Small footprint – message is local – scan and database is remote • Embedded • Tiny amount of resources required – scanning is done remotely
Can process 100s or even over 1000 messages per second Requests Bullet Signature updates every 1-10 minutes (only changes are downloaded) The SEI and SES are typically deployed on the same hardware The SEI is linked into the OEM application using C or C++ The SES runs as a Service or Daemon and it manages it’s own Database Updates The Database is usually between 3-10MB – will download a fresh DB upon startup if none present Star Engine – Enterprise(Very High Performance) Server or Appliance Hardware Linked Together by OEM at compile OEM Application C or C++ API Star Engine Interface (SEI) TCP / IP Star Engine Server (SES) (Service or Daemon) TCP / IP Mail-Filters Data Centers
Star Engine - Enterprise • The Star Engine Server is fully multi-threaded • The Star Engine Server will run as a Service under Windows or as a Daemon under Linux, FreeBSD, or Solaris • TCP/IP outbound on Port 80 is required – IP proxies are supported • Typical requirements are P4, 100MB RAM, Hard Disk optional • A unique Mail-Filters Customer ID is required to download the Bullet Signature Database
Star Engine – Desktop(Small Footprint) • Only requires 128kb of RAM • Can process 10s of messages per second • Secondary server can be anywhere, including and typically Mail-Filters’ Data Centers • Database updates are not required on the SEI (just the SES) • Same exact API as the Enterprise implementation • Can also be used in a server cluster environment – many SEI’s feeding one SES PC or Other Device (with limited resources) Linked Together by OEM at compile OEM Application C or C++ API Star Engine Interface TCP / IP Separate Server Star Engine Server TCP / IP Mail-Filters Data Centers
Email Server Star Engine – EmbeddedA Completely New Approach 4. Mail-Filters’ authenticates as the user to the ISP or Corporate email servers - the mail is delivered • Anti-Spam detection for edge devices with almost no resource requirements • OEM code requires less than 10kb of RAM • No software need be installed on any user PC – the service is turned on or off at the OEM device • Works with POP3 & IMAP • OEM device intercepts the message delivery request and sends it to Mail-Filters • Mail-Filters receives the messages on behalf of the end user, filters for viruses and spam, then sends the clean messages to the end user • OEM or customer determines what happens to spam (delete, mark with an X-header, decorate the subject line) • Since spam can be deleted and the downlink speed is probably slower than the link from Mail-Filters’ data centers to the email servers – good mail will get to the end user faster. 3, Mail-Filters makes the request on behalf of the user, filters the messages, then sends the good mail to the user. No mail is kept at Mail-Filters – it just passes through. WWW Mail-Filters Data Centers 2. OEM device intercepts the request based on port the request is made on (Ex. 110 = POP3) – and redirects the request to Mail-Filters’ data centers. 1. Email Client requests mail PC
Email Server Email Server Embedded Architecture Mail-Filters Data Centers The Internet OEM Device OEM Application Redirect Code Outbound Listening Code (Port 110 for POP3 or Port 147 for IMAP Requests) Customer Premise The Email Client requests email from an email server – it makes the request on port 110 or 147 – the OEM device redirects the request to Mail-Filters. A port is opened by the email server via Mail-Filters to the PC. The email is filtered, a policy is applied, then delivered to the Email Client. PCs
SDK Contents • Star Engine Server software executables • Star Engine Interface libraries in C and C++ • Simple Single-Threaded implementation example application • Documentation • Typical integration time is less than a day
Getting Started with the SDK • Install the Star Engine Server • Run the Star Engine Server • Run the Example Application • This application will scan the files in the directory of choice and all sub-directories to see if they are spam. The results will display on the screen. • Begin the Integration to the OEM application
The Star Engine API(The Star Engine Interface) • The Commands are Straight-Forward • Initialize – This command establishes a connection to the Star Engine Server • Shutdown – Used to tear down the thread after a successful Initialize command • Scan SMTP Buffer – Passes the SES the data to be scanned – will return TRUE if Spam • SCAN Buffer – Passes the SES data to be scanned – best used for non-SMTP types of content such as IM, SMS, web pages, etc. • Version – Returns the versions of all the components currently being used, including the database version date.
Testing Options • The Mail-Filters database is culled to eliminate old/unused signatures. • As a result, the catch rate will suffer on old corpuses of email • Best results are obtained with live (or very close to it) email. • There are several options to test the Mail-Filters technology • To test for catch rate or false positive rate • Use the Example scan utility to check individual messages in a directory • Send mail to an account Mail-Filters can set up for you at Cleantree.com. Good mail will go to the Inbox, spam to the Spam folder. Check results using your browser. • Integrate into the OEM application and run it to check catch rate. • To test throughput: • Unfortunately, the Example application is only a single-threaded application and will not show what the SES can achieve throughput-wise (it does fine on catch rate) • The only fair test is to do an integration and run email through it. Most OEMs fine the solution throughput is the same whether Mail-Filters technology is running or not. • To test Foreign Language: • Do a beta test with a customer or partner in the region of interest • Mail-Filters have several partners in various regions that may assist in a beta test, if desired.
Implementation Examples • Enterprise • Most OEMs have implemented the Mail-Filters technology as the primary anti-spam solution • AV solutions company scans for spam while it has the message in memory to scan for viruses. Because spam is more prevalent and is a much faster scan, spam is typically scanned for first. • Some have augmented their own anti-spam technology • Because Mail-Filters technology is both fast and accurate, some have used it as a pre-processor to their own, more computationally expensive technology, to increase the throughput of the overall solution, and to increase spam catch rates.
Implementation Examples • Desktop • Some devices don’t have the processing power or resources available for spam detection. For these, the Mail-Filters technology can provide a smaller footprint • Firewalls, security gateways, messaging gateways, enterprise PCs may prefer a secondary server to handle the scanning to free up resources on their own hardware. • An MSP has a cluster environment where there are many SEIs feeding one SES per tower. This is very efficient and allows their overall throughput to increase dramatically.
Implementation Examples • Embedded • Ideal for DSL routers, Cable Modems, Wireless gateways, SMB security gateways etc. • Because it requires no end user software installation or configuration, it is simple to sign-up and have spam and viruses eliminated.
Frequently Asked Questions • How do I get the SDK? • Sign the Mail-Filters MNDA and we’ll send it to you via email. • Is the Star Engine Server multi-threaded? • Yes. • Does it handle messages in double-byte character sets? • Yes, our technology catches spam in over 30 languages, including multi-byte character sets such as Japanese, Korean, Chinese, Arabic, and Hebrew. • How is the update interval set – can it be changed? • The update interval is set by the OEM, but can be changed on a customer by customer basis. The default is an incremental every 10 minutes and a full update written to disk once a week. • Will this solution work on less than a Pentium IV PC? • Yes, but it works more efficiently on a PIV.
Frequently Asked Questions • What happens if the SES can’t get a database, or quits running, or some other catastrophe? • The SES or SEI will fail safe. It will return a FALSE ( the message isn’t spam) and continue to process messages while trying to reconnect. The customer will see more missed spam, but won’t miss any messages. • What if the SES doesn’t have the rights to write the database to disk, or the disk is full? • The SES will continue to function properly and will acquire updates to the database in memory. The version command will return the database currently being used in RAM. • Is the API really just 5 functions? • Yes – it doesn’t get much simpler than that. • Can the SES return a probability of a message being spam? • No - Because the technology uses human editors to craft profiles and message signatures, we’re very very confident the message is spam if we identify it. Because our false positive rate is so low, our methodology is proven to be correct. A probability is required by technologies that guess or compute whether a message is spam – we know it, so we tell you. For those solutions that require a probability, they set our TRUE response to the highest probability – 10 or 1 or 100.
Conclusions • The Mail-Filters technology is easy to implement and provides options for any situation. • The underlying technology far surpasses what others are doing, giving the Mail-Filters OEM a significant advantage over competitors in catch rate and accuracy, language coverage, and throughput. • Human review provides the difference -the technology delivers it.