1 / 10

Jeremy Bingham, DailyKos

Jeremy Bingham, DailyKos.com. Initial Problems. Site tipped over way too often Required intensive administration Load times needed to be faster. MySQL Issues. Table design didn't scale Redesigned tables with more sensible primary keys Lots of MySQL 3.23isms everywhere

zach
Download Presentation

Jeremy Bingham, DailyKos

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jeremy Bingham, DailyKos.com

  2. Initial Problems • Site tipped over way too often • Required intensive administration • Load times needed to be faster

  3. MySQL Issues Table design didn't scale Redesigned tables with more sensible primary keys Lots of MySQL 3.23isms everywhere Re-engineered tables & code to take advantage of new features Had to stop assuming stuff was there for “a good reason”

  4. Caching and proxying • Started off with apache 1.3 as proxy • Cached front page & recent posts to disk every 5 minutes for anonymous readers • Caused problems during the Iowa caucuses when all servers updated their file caches simultaneously. • Memcached • Using since 2006 • Used even more in 2008

  5. Lighttpd as proxy • Spent weekend with K&R • Added memcached support to mod_magnet • Hacked up mod_magnet had issues with flexibility, also did way more than needed • Enter mod_mcpage • Only caches pages, nothing else • Gzips large pages • Works entirely in lighttpd • Greatly lowered stress on servers

  6. Hardware, before • Before: 10 dual Xeons w/ 2GB RAM, 2 dual Operons w/ 8GB RAM & 2 73 GB SCSI disks, image server, memcached server, combined search & SMTP server • Each webhead independent; all updates had to be applied to each server. • Inconvenient, nerve-wracking possibilities for error

  7. Hardware, after • After: 6 quad Xeons w/ 8GB RAM, SATA scratch disks, 2 octocore Xeons w/ 16 GB RAM, 2 73 GB SCSI for / and /tmp, 6x73GB RAID-10 w/ finely tuned XFS for DB files, consolidated image origin/search/SMTP, Xen VM for db backup, CDN for images • Webheads netboot with NFS root, run as cluster • Lower costs, better performance • New webheads pop right in

  8. DK Monthly Traffic, by Year

  9. How It Worked Out • Performance good overall • Unexpected issues w/ the Flash electoral map - 100 mbit switch no one noticed • Webhead load generally between 0.5 and 1.8 • Some slowdowns from ads and the map

More Related