240 likes | 591 Views
Anatomy of a Project: Cites Spam Control Mike Corn Director Security Services and Information Privacy Office of the CIO / CITES Topics Project anatomy CITES Spam Control deployment Q & A Project Timeline 2004 April 29th 2004 - First core team meeting
E N D
Anatomy of a Project: Cites Spam Control Mike Corn Director Security Services and Information Privacy Office of the CIO / CITES Mike Corn | CSC
Topics • Project anatomy • CITES Spam Control deployment • Q & A Mike Corn | CSC
Project Timeline 2004 • April 29th 2004 - First core team meeting • July 20th - End user testing (113 users) begins • Sept 17th - Focus group • Sept 28th - End user testing complete • Oct - RFP goes out • Nov - Vendor selected • Dec - Vendor negotiations • Dec 22nd - Contract signed (huzzah!) Mike Corn | CSC
Project Timeline 2005 • Jan 1st - Hardware ordered • Jan 31st - Hardware installed in data center • Feb 8th - Product installed • March 2nd - Antivirus in production • March 4th - Anti-spam available to techsupport • April 1st - System ‘lockdown’ • April 15th - Anti-spam in production • April 15th - May - Phased notification begins Mike Corn | CSC
Where could we have moved faster? • Start sooner • Throw more staff at it • Skip end-user testing • Accelerate vendor negotiations / contract signing Mike could have screwed around less Mike Corn | CSC
Project Structure • Involve those with experience in • Campus scale email • End-user support • Leverage our Project Management Framework as much as possible • Diverse team drawn Mike Corn | CSC
Core team members and location w/in CITES • Production Applications Group • Beth Engelbrecht-Wiggans • Warner Brigham • Systems Management Group • Ben O’Connor • Jim Pirzyk • Help Desk • Mark Zinzow • Documentation • Kate Ower • ISE - Leslie Sherman • CITES Project Management - Tim Carroll • Strategic Communications - Brian Mertz • Security - Leslie Rankin - Bob Foertsch Mike Corn | CSC
Annoying Issues • DNS upgrade (from 1 or 2 lookups per message to 5 to 6) • Jan 6th - discover unusual power requirements for database servers (resolved 1/26) • Feb 10th - discover “help desk” access is not granular enough • March 4th & 14th - Milter tuning Mike Corn | CSC
More Interesting Issues • Vender viability • Expectation management • Automated content review • Virus handling policy • Default spam handling policy • ADA Compliance • Non-people accounts • Global whitelist / blacklist • Official communications • Help Desk / support issues • Quarantine size (spam retention) • Backup strategy • Interactions with other mail systems / antispam systems • Netid changes, Catastrophic failure, Lifetime forwarding, etc………… Mike Corn | CSC
Scaling Issues • Now it gets ugly Mike Corn | CSC
Hardware • Sun v210 * 8 (front ends) • Sun v490 * 2 (database server) • Foundry ServerIron XL’s (load balancers * 2) • SAN based quarantine (500 gigs) • Inexpensive disk dump array (1 terabyte) • Sun v210 * 3 (test / dev system) Mike Corn | CSC
Email Relays w/out CSC Internet Need more capacity? Debugging an email problem? Mike Corn | CSC
Email Relays w/CSC Internet CSC Relay Mike Corn | CSC
Who Managed ‘Email’? Internet The distant past (~2004) Relays DS Exchange Express Dept. Mail Servers Mike Corn | CSC
Who Manages ‘email’? CSC Internet From now on Relays DS Exchange SLB SSL Accel. Express Dept. Mail Servers Express-SMTP Mike Corn | CSC
The Organic Network • Nothing is stand-alone • Success requires collaboration not consensus • You knew this before we did Mike Corn | CSC
CSC Deployment • April 15th (tomorrow!) full production • tag & deliver for all email • Phased deployment • done to manage support not system load • 4/15 all tech support invited to activate • 4/22 all acpros and civil service staff • 4/26 all faculty • 4/28 computer security day (faculty) • grad / undergrad at week intervals Mike Corn | CSC
CSC Support • CITES Help Desk • Help Desk interface will be rolled out to technical support (similar to password registrar functionality) • DS Exchange subscribers will receive memo from Departmental Services Mike Corn | CSC
Training • Around 400 faculty and staff have attended a training session • PowerPoint presentation on Security archives (https://www-s.cites-security.uiuc.edu/archive/index.php) • Training booklet also on Security archives • More free training (http://training.cites.uiuc.edu/spam/default.htm) • Computer Security Day: April 28th 10am - 3pm Illini Union Room C Mike Corn | CSC
Help Desk Interface Mike Corn | CSC
CSC Phase II • How to support department mail servers • Version 3+ of software • Non-people accounts • Revisit provisioning strategy • Possible web access to quarantine • Pager spam • Outbound SMTP virus filtering Phase II implies issues to be examined – not promised! Mike Corn | CSC
Q & A • Whatever you ask here, stays here. Out of shame. Mike Corn | CSC
Last Fall’s Slide Some misc. capacity requirements 1.5 million messages per day Peak > 100k per hour Average message size = 26k Use a multiplier of 2-3 for peaks/load during a virus outbreak Appliance Internet Relays • Viruses deleted • SPAM quarantined Express Mike Corn | CSC
Why Load Balancers? Mike Corn | CSC