My Drupal Blog Web Server Optimization - Part 2

Printer-friendly versionPDF version

The change that I will describe here was a direct consequence of load testing. I first stress tested the server with 5 users and than with 10 users. Till 10 users the server was just eating up the load. However with 25 users for about an hour and half it looked like a different story. You can have a look at the results yourself in this article here and the analysis. The 3rd test was done after the changes described below and it looked pretty upbeat for the 25 users for almost a couple of hours of stress testing.

First thing to understand is what your Apache server using to serve the site. In Debian it was "prefork" but it could also be "MPM" worker module. So my description is for the prefork module. This is a thread safe implementation which is the correct choice if you are using php based implementations. The other MPM worker module is a better choice for static pages. Anyways as su or root if you execute the below you can see what your Apache server is using. I removed some extra unneeded output.

/usr/sbin/apache2 -V
Server version: Apache/2.x.xx(Debian)
Server MPM:     Prefork
  threaded:     no
    forked:     yes (variable process count)
Server compiled with....
 -D APACHE_MPM_DIR="server/mpm/prefork"

Having figured out what module you are using, you know now what section of the apache.conf file you have to tune. This is what the default prefork implementation looked like before tuning :

# prefork MPM
# StartServers: number of server processes to start
# MinSpareServers: minimum number of server processes which are kept spare
# MaxSpareServers: maximum number of server processes which are kept spare
# MaxClients: maximum number of server processes allowed to start
# MaxRequestsPerChild: maximum number of requests a server process serves

    StartServers          5
    MinSpareServers       5
    MaxSpareServers      10
    MaxClients          150
    MaxRequestsPerChild   0

First impulse was to just edit this in place, but we must do it properly if we are doing it at all. So in Debian there is a /etc/apache2/conf.d directory that the apache.conf file sources. This is the correct place for all customizations. So if an apache upgrade provide a new apache.conf file your changes will be preserved. So to that end I created a new file in that directory. Feel free to call it anything. And in that I put the over-ridden values as below:

    StartServers          10
    MinSpareServers       10
    MaxSpareServers      15
    MaxClients          50
    MaxRequestsPerChild   10000

Lets just start with the value MaxClients. This is the number of apache process I could afford to start on my server without exhausting out memory for all other processes. You can find out on average how much your apache2 processes require by executing the following command

ps aux | grep 'apache2' | awk '{print $6/1024 " MB";}'

I had apache2 process pretty much round about the 20MB figure. So thats what I took and approximately took half of my ram i.e. 1024MB and divided it by 20 to come to the 50 MaxClients value. The value of 150 was just way too much. This was the probable cause of so less throughput in my 2nd stress test mentioned in my stress test article here.

Having done that the other values were non-trivial. I was ok with having my server start already with 10 threads, keep around a minimum of 10 spare servers or a maximum of 15 spare servers.

The important thing I think is the "MaxRequestsPerChild" value. It sounds misleading in what it says. The actual meaning is the number of request a child apache process will handle before being destroyed and re-created. I think this is important as the logic behind that is that if there are any memory leaks by implementations, that memory is reclaimed when process is destroyed and recreated. Just kept this value intuitively high. Too small a value is also harmful.

So happy initial tuning to you all as well. There are more room for improvements. But this is how far I will go currently as I have some other infrastructure issues to address. And then we will revisit Web server optimization with some cache solution. Till than take care.


Top level category: