Shoehorning Apache Onto Your Box

1. Shoehorning Apache Onto Your Box System Sizing Tips Sander Temme <sander@temme.net>

2. So, your web server machine may not be the fastest, shiniest machine,but it can still take a few hits without going down.

3. Assumptions • Limited Resources • Work With What You Have • You’re probably running Linux • No Compiling or Recompiling! • You can find httpd.conf

4. Purposes of Monitoring • Observation • Extrapolation • Signals/Alerts • Testing

5. Monitoring Your Server

6. Monitoring Apache

7. Monitoring: Nagios

8. Other Monitoring Tools • vmstat • top • free

9. Web Server Logs • ErrorLog • LogLevel: debug, info, notice, warn, error, crit • Access Log: TransferLog or CustomLog • Common Log Format

10. Common Log Format 172.16.2.91 - - [16/Nov/2003:15:23:27 -0800] "GET /~sctemme/ HTTP/1.1" 200 1446 172.16.2.91 - - [16/Nov/2003:15:23:28 -0800] "GET /~sctemme/css/doc.css HTTP/1.1" 200 167 172.16.2.91 - - [16/Nov/2003:15:23:28 -0800] "GET /~sctemme/css/menu.css HTTP/1.1" 200 623 172.16.2.91 - - [16/Nov/2003:15:23:29 -0800] "GET /favicon.ico HTTP/1.1" 404 283 172.16.2.91 - - [16/Nov/2003:15:23:34 -0800] "GET /~sctemme/index.html HTTP/1.1" 200 1446

11. Configuring for Performance • Configuring Apache • Tuning the Operating System • Scaling Your Site

12. Apache Configuration • Process/Thread Management • DNS Lookups • Avoid .htaccess Files • Disable unused modules • Tune your App Tier • Cache if you Can

13. DNS Lookups • HostnameLookups • Access Control • Bad: Deny from example.com • Good: Deny from 172.160.234.5

14. .htaccess Files • Per-directory configuration files • Accessed for every request • Best performance: AllowOverride none GET /dir1/dir2/restricted.html HTTP/1.0

15. MaxClients • Configuration file directive • Maximum number of workers • Apache 1.3, 2.0 Prefork: processes • Apache 2.0 Worker: threads * processes • Limit according to resources (memory)

16. Sizing MaxClients • Take total RAM • Subtract OS allowance • look at free value without Apache, etc. • Subtract external program allowance • JVM, cgi programs, MySQL? • Divide by httpd process size • Read process size from top

17. Top

18. Tune your App Tier • Tomcat • Edit server.xml, tune minProcessors, maxProcessors • Tune JVM (Heap, Garbage Collection) • MySQL • Ships with various scenarios in support-files: • my-{small,medium,large,huge}.conf • PHP & prefork: every child makes a connection

19. Cache your Content • Dynamic Content is Expensive • Static Content is Cheap • Don’t Regenerate the Same Page • Cache it! • Wiki, Blog, Catalogs, … • Example: wiki.apache.org

20. System Tuning Tips • RAM and swap space • ulimit: files and processes • Turn off unused services and modules • Patch your OS

21. RAM and Swap • Swap is disk-based Extension of RAM • Excessive swapping kills performance • Tune MaxClients • Never have more memory than swap • Upgrade RAM -> add more swap space

22. ulimit • Per-process resource limits • Built-in command of sh, bash • Important limits: • processes (-u) • open files (-n) • Set in invoking shell • Code in Apache 2.0 startup script • ulimit -S -n `ulimit -H -n` • Linux: /etc/security/limits.conf

23. Example: ajax.apache.org • 4xItanium CPU HP Integrity Server, 8Gb RAM • RHEL 3.0 • www.apache.org, most TLPs, JIRA, Bugzilla, Wiki • Very beefy machine • Does not perform well • Especially disk access • Upgrade to RHEL 4.0? (future) • JIRA Upgrade Helped • Cache for Wiki Helped • Wiki under mod_pyton? (future?) • …

24. Scaling Your Site • Vertically • Tiered Model • Horizontally • Load Balancing

25. Scaling Vertically TCP/IP Client TCP/IP TCP/IP

26. Scaling Vertically • Move Services to Other Hosts • Pros: • Less resource contention • Specialized hardware • Scale tiers individually • Cons: • Development/Deployment harder • More hosts to manage

27. Scaling Horizontally Client

28. Tips • Observe Before You Act • Act on Monitoring Results • Don’t Overload Your System • Use A Staging Server

29. Q&A

30. Further Reading • Ryan B. Bloom, Apache Server 2.0: The Complete Reference, 2002 McGraw Hill Osborne; ISBN 0-07-222344-8 • Ben Laurie and Peter Laurie, Apache: The Definitive Guide (3rd Edition), 2002 O’Reilly & Associates; ISBN 0596002033 • Patrick Killelea, Web Performance Tuning, 2nd Edition, 2002 O’Reilly & Associates; ISBN 0-596-00172-X • http://httpd.apache.org/docs-2.0/misc/perf-tuning.html • http://httpd.apache.org/docs/misc/perf-tuning.html • Adrian Cockcroft, Richard Pettit and Sun Microsystems Press, Sun Performance and Tuning: Java and the Internet (2nd Edition), 1998 Prentice Hall PTR; ISBN 0130952494 • Ken Coar and Rich Bowen, Apache Cookbook, 2003 O’Reilly & Associates; ISBN 0596001916

31. Conference Roadmap • T20: Scalable Internet Architectures • WE04: QoS management of Internet services • WE09: Troubleshooting Apache configurations • TH01: Scaling Apache 2.x to > 20,000 concurrent downloads • TH17: Caching, Tips for Improving Performance • TH21: Powering High-volume web sites with Lenya/Cocoon and mod_cache • FR05: Improving Web Performance with Dynamic Compression • FR09: Clustering and load balancing using mod_proxy

32. Current Version http://apache.org/~sctemme/ApconEU2005/WE05/

33. Thank You

34. Backup Slides

35. Selecting Your MPM • Apache 2.0 only! • Processes and Threads • Differences between platforms • Thread-safety issues

36. Processes and Threads • Process: • Own copy of data structures • Shares: program code, shared memory • Context switches expensive • Thread: • Runs within process • Shares process environment • No context switch

37. Platforms and Threading • Context switches expensive on Solaris, AIX • Context switches cheaper on Linux • Solaris uses M:N threading • Linux uses 1 process per thread • LinuxThreads implementation is old • Replaced by Native Posix Thread Library (NPTL) in 2.6 • NTPL already in RH 9, RHAS 3

38. Thread-safety • Third-party modules and libraries • mod_perl: experimental threading in Perl 5.6; more mature in Perl 5.8 • PHP: uses many third-party libraries • FreeBSD: threading not reliable until 5.x • Use KSE threading in 5.x • Still not endorsed by ASF