Ubuntu 16.04 Apache 2.4 with PHP-FPM

PHP-FPM does have some advantages depending on the solution and the common path is to use Nginx with PHP-FPM. However what happens when you want to utilize the normal features of Apache, such as basics like .htaccess files, but still keep the tuning options open that come with PHP-FPM? Well, there is a module for that!

This guide is going to assume a fresh Ubuntu 16.04 server to illustrate everything from start to finish, and will assume that all sites on this server will use the same php-fpm pool.

First, installed the required packages for your web server:

[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install php7.0-fpm apache2

Now confirm or update the Apache configuration to use the mpm_event_module instead of the mpm_prefork_module:

[[email protected] ~]# a2enmod actions
[[email protected] ~]# apache2ctl -M | grep mpm
[[email protected] ~]# a2dismod mpm_prefork
[[email protected] ~]# a2dismod mpm_worker
[[email protected] ~]# a2enmod mpm_event

Then tell Apache to send all PHP requests over to PHP-FPM by creating a new configuration file:

[[email protected] ~]# vim /etc/apache2/conf-available/php.conf
<FilesMatch \.php$>
	SetHandler "proxy:unix:/run/php/php7.0-fpm.sock|fcgi://localhost/"
</FilesMatch>

Enable the new Apache PHP configuration:

[[email protected] ~]# a2enconf php.conf

Confirm PHP-FPM is set to use sockets instead of TCP connections for performance purposes, and also confirm the following additional settings:

[[email protected] ~]# vim /etc/php/7.0/fpm/pool.d/www.conf
; listen = 127.0.0.1:9000
listen = /run/php/php7.0-fpm.sock
...
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
user = www-data
group = www-data

Enable FCGI proxy, then restart Apache and PHP-FPM to apply the changes above:

[[email protected] ~]# a2enmod proxy_fcgi
[[email protected] ~]# systemctl restart apache2
[[email protected] ~]# systemctl restart php7.0-fpm

If you are using a software firewall on the server, open ports 80/443 accordingly. This example will open them up to the world. Adjust yours accordingly:

[[email protected] ~]# ufw allow 80
[[email protected] ~]# ufw allow 443

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at x.x.x.x/info.php:

[[email protected] ~]# vim /var/www/html/info.php
<?php phpinfo(); ?>

And your done!

Using multiple PHP-FPM pools

What happens if you want to isolate each site to their own PHP-FPM pool instead of using a shared pool? That is easy enough to do. Assuming that you followed everything in this guide to get to this point, do the following.

First, disable the global Apache configuration for PHP:

[[email protected] ~]# a2disconf php.conf

Create a new PHP-FPM pool for this specific site and update it accordingly:

[[email protected] ~]# cp /etc/php/7.0/fpm/pool.d/www.conf /etc/php/7.0/fpm/pool.d/example.com.conf
[[email protected] ~]# vim /etc/php/7.0/fpm/pool.d/example.com.conf
; Start a new pool named 'www'.
; the variable $pool can be used in any directive and will be replaced by the
; pool name ('www' here)
[example.com]
...
; listen = 127.0.0.1:9000
listen = /run/php/www.example.com-php7.0-fpm.sock
...
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
user = www-data
group = www-data

Then update the site’s Apache vhost to point to a new PHP-FPM pool in both the 80 and 443 stanzas. Be sure to update the socket accordingly for your site in the 2 sections below! (ie: unix:/run/php/www.example.com-php7.0-fpm.sock)

[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf
<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

	# Send PHP requests to php-fpm
        <FilesMatch \.php$>
                SetHandler "proxy:unix:/run/php/www.example.com-php7.0-fpm.sock|fcgi://localhost/"
        </FilesMatch>

...
<VirtualHost *:443>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

	# Send PHP requests to php-fpm
        <FilesMatch \.php$>
                SetHandler "proxy:unix:/run/php/www.example.com-php7.0-fpm.sock|fcgi://localhost/"
        </FilesMatch>
...

Enable FCGI proxy, then restart Apache and PHP-FPM to apply the changes above:

[[email protected] ~]# a2enmod proxy_fcgi
[[email protected] ~]# systemctl restart php7.0-fpm
[[email protected] ~]# systemctl restart apache2

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at example.com/info.php:

[[email protected] ~]# vim /var/www/vhosts/example.com/info.php
<?php phpinfo(); ?>

And your done!

Ubuntu 14.04 Apache 2.4 with PHP-FPM

PHP-FPM does have some advantages depending on the solution and the common path is to use Nginx with PHP-FPM. However what happens when you want to utilize the normal features of Apache, such as basics like .htaccess files, but still keep the tuning options open that come with PHP-FPM? Well, there is a module for that!

This guide is going to assume a fresh Ubuntu 14.04 server to illustrate everything from start to finish, and will assume that all sites on this server will use the same php-fpm pool.

First, installed the required packages for your web server:

[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install php5-fpm apache2 libapache2-mod-fastcgi

Now update the Apache configuration to use the mpm_event_module instead of the mpm_prefork_module:

[[email protected] ~]# a2enmod actions
[[email protected] ~]# apache2ctl -M | grep mpm
[[email protected] ~]# a2dismod mpm_prefork
[[email protected] ~]# a2dismod mpm_worker
[[email protected] ~]# a2enmod mpm_event

Then tell Apache to send all PHP requests over to PHP-FPM by creating a new configuration file:

[[email protected] ~]# vim /etc/apache2/conf-available/php.conf

<IfModule mod_fastcgi.c>
        AddHandler php5.fcgi .php
        Action php5.fcgi /php5.fcgi
        Alias /php5.fcgi /usr/lib/cgi-bin/php5.fcgi
        FastCgiExternalServer /usr/lib/cgi-bin/php5.fcgi -socket /var/run/php-fpm.sock -pass-header Authorization -idle-timeout 3600
        <Directory /usr/lib/cgi-bin>
                Require all granted
        </Directory>
</IfModule>

Enable the new Apache PHP configuration:

[[email protected] ~]# a2enconf php.conf

Confirm PHP-FPM is set to use sockets instead of TCP connections for performance purposes, and also confirm the following additional settings:

[[email protected] ~]# vim /etc/php5/fpm/pool.d/www.conf
; listen = 127.0.0.1:9000
listen = /var/run/php-fpm.sock
...
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
user = www-data
group = www-data

Restart Apache and PHP-FPM to apply the changes:

[[email protected] ~]# service apache2 restart
[[email protected] ~]# service php5-fpm restart

If you are using a software firewall on the server, open ports 80/443 accordingly. This example will open them up to the world. Adjust yours accordingly:

[[email protected] ~]# ufw allow 80
[[email protected] ~]# ufw allow 443

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at x.x.x.x/info.php:

[[email protected] ~]# vim /var/www/html/info.php
<?php phpinfo(); ?>

And your done!

Using multiple PHP-FPM pools

What happens if you want to isolate each site to their own PHP-FPM pool instead of using a shared pool? That is easy enough to do. Assuming that you followed everything in this guide to get to this point, do the following.

First, disable the global Apache configuration for PHP:

[[email protected] ~]# a2disconf php.conf

Create a new PHP-FPM pool for this specific site and update it accordingly:

[[email protected] ~]# cp /etc/php5/fpm/pool.d/www.conf /etc/php5/fpm/pool.d/example.com.conf
[[email protected] ~]# vim /etc/php5/fpm/pool.d/example.com.conf
; listen = 127.0.0.1:9000
listen = /var/run/www.example.com-php5-fpm.sock
...
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
user = www-data
group = www-data

Then update the site’s Apache vhost to point to a new PHP-FPM pool in both the 80 and 443 stanzas. Be sure to update the socket accordingly for your site in the 2 sections below! (ie: -socket /var/run/www.example.com-php5-fpm.sock)

[[email protected] ~]# vim /etc/apache2/sites-enabled/example.com.conf
<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

	# Send PHP requests to php-fpm
	<IfModule mod_fastcgi.c>
		AddHandler php5.fcgi .php
		Action php5.fcgi /php5.fcgi
		Alias /php5.fcgi /usr/lib/cgi-bin/php5.fcgi
		FastCgiExternalServer /usr/lib/cgi-bin/php5.fcgi -socket /var/run/www.example.com-php5-fpm.sock -pass-header Authorization -idle-timeout 3600
		<Directory /usr/lib/cgi-bin>
			Require all granted
		</Directory>
	</IfModule>
...

<VirtualHost *:443>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

	# Send PHP requests to php-fpm
	<IfModule mod_fastcgi.c>
		AddHandler php5.fcgi .php
		Action php5.fcgi /php5.fcgi
		Alias /php5.fcgi /usr/lib/cgi-bin/php5.fcgi
		FastCgiExternalServer /usr/lib/cgi-bin/php5.fcgi -socket /var/run/www.example.com-php5-fpm.sock -pass-header Authorization -idle-timeout 3600
		<Directory /usr/lib/cgi-bin>
			Require all granted
		</Directory>
	</IfModule>
...

Then restart the services:

[[email protected] ~]# systemctl restart php5-fpm
[[email protected] ~]# systemctl restart apache2

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at example.com/info.php:

[[email protected] ~]# vim /var/www/vhosts/example.com/info.php
<?php phpinfo(); ?>

And your done!

CentOS 7 Apache 2.4 with PHP-FPM

PHP-FPM does have some advantages depending on the solution and the common path is to use Nginx with PHP-FPM. However what happens when you want to utilize the normal features of Apache, such as basics like .htaccess files, but still keep the tuning options open that come with PHP-FPM? Well, there is a module for that!

This guide is going to assume a fresh CentOS 7 server to illustrate everything from start to finish, and will assume that all sites on this server will use the same php-fpm pool.

First, installed the required packages for your web server:

[[email protected] ~]# yum install httpd httpd-tools mod_ssl php-fpm

Now update the Apache configuration to use the mpm_event_module instead of the mpm_prefork_module:

[[email protected] ~]# vim /etc/httpd/conf.modules.d/00-mpm.conf 
# LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
LoadModule mpm_worker_module modules/mod_mpm_worker.so

Then tell Apache to send all PHP requests over to PHP-FPM by creating a new configuration file:

[[email protected] ~]# vim /etc/httpd/conf.d/php.conf

# Tell the PHP interpreter to handle files with a .php extension.

# Proxy declaration
<Proxy "unix:/var/run/php-fpm/default.sock|fcgi://php-fpm">
	# we must declare a parameter in here (doesn't matter which) or it'll not register the proxy ahead of time
    	ProxySet disablereuse=off
</Proxy>

# Redirect to the proxy
<FilesMatch \.php$>
	SetHandler proxy:fcgi://php-fpm
</FilesMatch>

#
# Allow php to handle Multiviews
#
AddType text/html .php

#
# Add index.php to the list of files that will be served as directory
# indexes.
#
DirectoryIndex index.php

#
# Uncomment the following lines to allow PHP to pretty-print .phps
# files as PHP source code:
#
#<FilesMatch \.phps$>
#	SetHandler application/x-httpd-php-source
#</FilesMatch>

Tweak PHP-FPM to use sockets instead of TCP connections for performance purposes as follows:

[[email protected] ~]# vim /etc/php-fpm.d/www.conf
; listen = 127.0.0.1:9000
listen = /var/run/php-fpm/default.sock
...
listen.allowed_clients = 127.0.0.1
listen.owner = apache
listen.group = apache
listen.mode = 0660
user = apache
group = apache

And lastly, enable the services to start on boot and start them up:

[[email protected] ~]# systemctl enable php-fpm
[[email protected] ~]# systemctl enable httpd
[[email protected] ~]# systemctl start php-fpm
[[email protected] ~]# systemctl start httpd

If you are using a software firewall on the server, open ports 80/443 accordingly. This example will open them up to the world. Adjust yours accordingly:

[[email protected] ~]# firewall-cmd --zone=public --permanent --add-service=http
[[email protected] ~]# firewall-cmd --zone=public --permanent --add-service=https
[[email protected] ~]# firewall-cmd --reload

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at x.x.x.x/info.php:

[[email protected] ~]# vim /var/www/html/info.php
<?php phpinfo(); ?>

And your done!

Using multiple PHP-FPM pools

What happens if you want to isolate each site to their own PHP-FPM pool instead of using a shared pool? That is easy enough to do. Assuming that you followed everything in this guide to get to this point, do the following.

First, disable the global Apache configuration for PHP:

[[email protected] ~]# mv /etc/httpd/conf.d/php.conf /etc/httpd/conf.d/php.conf.bak

Create a new PHP-FPM pool for this specific site and update it accordingly:

[[email protected] ~]# cp /etc/php-fpm.d/www.conf /etc/php-fpm.d/example.com.conf
[[email protected] ~]# vim /etc/php-fpm.d/example.com.conf
; listen = 127.0.0.1:9000
listen = /var/run/php-fpm/example.com.sock
...
listen.allowed_clients = 127.0.0.1
listen.owner = apache
listen.group = apache
listen.mode = 0660
user = apache
group = apache

Then update the site’s Apache vhost to point to a new PHP-FPM pool in both the 80 and 443 stanzas. Be sure to update the socket accordingly for your site in the 2 sections below! (ie: unix:/var/run/php-fpm/example.com.sock)

[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf
<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

        # Proxy declaration
        <Proxy "unix:/var/run/php-fpm/example.com.sock|fcgi://php-fpm">
                # we must declare a parameter in here (doesn't matter which) or it'll not register the proxy ahead of time
                ProxySet disablereuse=off
                # Note: If you configure php-fpm to use the "ondemand" process manager, then use "ProxySet disablereuse=on"
        </Proxy>

        # Redirect to the proxy
        <FilesMatch \.php$>
                SetHandler proxy:fcgi://php-fpm
        </FilesMatch>
...
<VirtualHost *:443>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

        # Proxy declaration
        <Proxy "unix:/var/run/php-fpm/example.com.sock|fcgi://php-fpm">
                # we must declare a parameter in here (doesn't matter which) or it'll not register the proxy ahead of time
                ProxySet disablereuse=off
                # Note: If you configure php-fpm to use the "ondemand" process manager, then use "ProxySet disablereuse=on"
        </Proxy>

        # Redirect to the proxy
        <FilesMatch \.php$>
                SetHandler proxy:fcgi://php-fpm
        </FilesMatch>
...

Then restart the services:

[[email protected] ~]# systemctl restart php-fpm
[[email protected] ~]# systemctl restart httpd

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at example.com/info.php:

[[email protected] ~]# vim /var/www/vhosts/example.com/info.php
<?php phpinfo(); ?>

And your done!

CentOS 6 Apache 2.4 with PHP-FPM

PHP-FPM does have some advantages depending on the solution and the common path is to use Nginx with PHP-FPM. However what happens when you want to utilize the normal features of Apache, such as basics like .htaccess files, but still keep the tuning options open that come with PHP-FPM? Well, there is a module for that!

This guide is going to assume a fresh CentOS 6 server to illustrate everything from start to finish, and will assume that all sites on this server will use the same php-fpm pool.

Apache 2.2 has no native modules for working with fastcgi. So the options would be to install mod_fastcgi from source or use a older SRPM from repos that may not be too well known or maintained. As both those options are less than ideal, we will be installing Apache 2.4 from the IUS repository to avoid the patch management issues associated with source installations.

First, install the repos needed for the updated packages:

[[email protected] ~]# rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
[[email protected] ~]# rpm -ivh https://dl.iuscommunity.org/pub/ius/stable/CentOS/6/x86_64/ius-release-1.0-15.ius.centos6.noarch.rpm

Then install the required packages for your web server:

[[email protected] ~]# yum install httpd24u php56u-fpm

Now update the Apache configuration to use the mpm_event_module instead of the mpm_prefork_module:

[[email protected] ~]# vim /etc/httpd/conf.modules.d/00-mpm.conf 
# LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
LoadModule mpm_worker_module modules/mod_mpm_worker.so

Then tell Apache to send all PHP requests over to PHP-FPM by creating a new configuration file:

[[email protected] ~]# vim /etc/httpd/conf.d/php.conf

# Tell the PHP interpreter to handle files with a .php extension.

<Proxy "unix:/var/run/php-fpm/default.sock|fcgi://php-fpm">
	# we must declare a parameter in here (doesn't matter which) or it'll not register the proxy ahead of time
	# Note: If you configure php-fpm to use the "ondemand" process manager, then use "ProxySet disablereuse=on"
	ProxySet disablereuse=off
</Proxy>

# Redirect to the proxy
<FilesMatch \.php$>
	SetHandler proxy:fcgi://php-fpm
</FilesMatch>

Tweak PHP-FPM to use sockets instead of TCP connections for performance purposes as follows:

[[email protected] ~]# vim /etc/php-fpm.d/www.conf
; listen = 127.0.0.1:9000
listen = /var/run/php-fpm/default.sock
...
listen.owner = apache
listen.group = apache
listen.mode = 0660
user = apache
group = apache

Enable the services to start on boot and start them up:

[[email protected] ~]# chkconfig php-fpm on
[[email protected] ~]# chkconfig httpd on
[[email protected] ~]# service php-fpm start
[[email protected] ~]# service httpd start

If you are using a software firewall on the server, open ports 80/443 accordingly. This example will open them up to the world. Adjust yours accordingly:

[[email protected] ~]# vim /etc/sysconfig/iptables
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT 
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT 
[[email protected] ~]# service iptables restart

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at x.x.x.x/info.php:

[[email protected] ~]# vim /var/www/html/info.php
<?php phpinfo(); ?>

And your done!

Using multiple PHP-FPM pools

What happens if you want to isolate each site to their own PHP-FPM pool instead of using a shared pool? That is easy enough to do. Assuming that you followed everything in this guide to get to this point, do the following.

First, disable the global Apache configuration for PHP:

[[email protected] ~]# mv /etc/httpd/conf.d/php.conf /etc/httpd/conf.d/php.conf.bak

Create a new PHP-FPM pool for this specific site and update it accordingly:

[[email protected] ~]# cp /etc/php-fpm.d/www.conf /etc/php-fpm.d/example.com.conf
[[email protected] ~]# vim /etc/php-fpm.d/example.com.conf
; Start a new pool named 'www'.
; the variable $pool can we used in any directive and will be replaced by the
; pool name ('www' here)
[example.com]
...
; listen = 127.0.0.1:9000
listen = /var/run/php-fpm/example.com.sock
...
listen.owner = apache
listen.group = apache
listen.mode = 0660
user = apache
group = apache

Then update the site’s Apache vhost to point to a new PHP-FPM pool in both the 80 and 443 stanzas. Be sure to update the socket accordingly for your site in the 2 sections below! (ie: unix:/var/run/php-fpm/example.com.sock)

[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf
<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

	# Send PHP requests to php-fpm
	<Proxy "unix:/var/run/php-fpm/example.com.sock|fcgi://php-fpm">
		# we must declare a parameter in here (doesn't matter which) or it'll not register the proxy ahead of time
		# Note: If you configure php-fpm to use the "ondemand" process manager, then use "ProxySet disablereuse=on"
		ProxySet disablereuse=off
	</Proxy>

	# Redirect to the proxy
	<FilesMatch \.php$>
		SetHandler proxy:fcgi://php-fpm
	</FilesMatch>
...
<VirtualHost *:443>
        ServerName example.com
        ServerAlias www.example.com
        DocumentRoot /var/www/vhosts/example.com

	# Send PHP requests to php-fpm
	<Proxy "unix:/var/run/php-fpm/example.com.sock|fcgi://php-fpm">
		# we must declare a parameter in here (doesn't matter which) or it'll not register the proxy ahead of time
		# Note: If you configure php-fpm to use the "ondemand" process manager, then use "ProxySet disablereuse=on"
		ProxySet disablereuse=off
	</Proxy>

	# Redirect to the proxy
	<FilesMatch \.php$>
		SetHandler proxy:fcgi://php-fpm
	</FilesMatch>
...

Then restart the services:

[[email protected] ~]# service php-fpm restart
[[email protected] ~]# service httpd restart

Finally, test a site to ensure PHP is working and is using PHP-FPM by creating the file below, then visiting the page at example.com/info.php:

[[email protected] ~]# vim /var/www/vhosts/example.com/info.php
<?php phpinfo(); ?>

And your done!

Apache quick stats

When troubleshooting performance issues with Apache that happened earlier in the day or week, it is very useful to parse the logs quickly to determine quick facts about the inbound connection rates. It may reveal a period of increased traffic that needs to be investigated further, therefore giving you that thread to begin unraveling the problem.

To get the total connections per day for a website, run the following:

[[email protected] ~]# cat /var/log/httpd/www.example.com-access.log | awk '{print $4}' | cut -d: -f1 |uniq -c
   1247345 [20/Feb/2017
   1331908 [21/Feb/2017
   1295677 [22/Feb/2017
   1435275 [23/Feb/2017
   1023423 [24/Feb/2017
   1342332 [25/Feb/2017
   1293422 [26/Feb/2017
   2131198 [27/Feb/2017

To get the total connections per day for each website on the server, run the following:

[[email protected] ~]# for i in `ls /var/log/httpd/*-access.log`; do echo $i && cat $i | awk '{print $4}' | cut -d: -f1 |uniq -c && echo ""; done
/var/log/httpd/www.example.com-access.log
   1247345 [20/Feb/2017
   1331908 [21/Feb/2017
   1295677 [22/Feb/2017
   1435275 [23/Feb/2017
   1023423 [24/Feb/2017
   1342332 [25/Feb/2017
   1293422 [26/Feb/2017
   2131198 [27/Feb/2017

/var/log/httpd/www.example02.com-access.log
   2542 [20/Feb/2017
   7586 [21/Feb/2017
   4776 [22/Feb/2017
   2975 [23/Feb/2017
  16756 [24/Feb/2017
   9874 [25/Feb/2017
   1638 [26/Feb/2017
   9654 [27/Feb/2017

To get the connections per hour for a specific day, run the following:

[[email protected] ~]# grep "27/Feb" /var/log/httpd/www.example.com-access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":00"}' | sort -n | uniq -c
  50205 03:00
  90516 04:00
  64837 05:00
  47410 06:00
  44876 07:00
  41098 08:00
  38996 09:00
  37234 10:00
  43704 11:00
  58702 12:00
  58922 13:00
  72592 14:00
  77792 15:00
  88882 16:00
  80815 17:00
  98287 18:00
 617857 19:00
  90507 20:00
  98568 21:00
 147584 22:00
 181814 23:00

Based off that output, there was a massive spike in connections during the 9:00PM hour (19:00). So now lets break the 9:00PM hour down to show the connections per minute:

[[email protected] ~]# grep "27/Feb/2017:19" /var/log/httpd/www.example.com-access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'
   1629 19:00
   1664 19:01
   1840 19:02
  10493 19:03
  13728 19:04
  17608 19:05
   1377 19:06
   2333 19:07
   1980 19:08
   2056 19:09
   2123 19:10
...
   1997 19:57
   1631 19:58
   1988 19:59

As shown above, there was some sort of traffic spike that occurred between 9:03PM – 9:05PM. As the window has been narrowed down to a 3 minute period, more specific analysis can be performed. The examples below will focus on what was happening around 9:03PM.

To list the top 10 IP’s accessing the site during around 9:03PM

[[email protected] ~]# grep "27/Feb/2017:19:03" /var/log/httpd/www.example.com-access.log | awk '{print $1}' | sort -nr | uniq -c |sort -nr | head

To list the top most called elements on the site:

[[email protected] ~]# grep "27/Feb/2017:19:03" /var/log/httpd/www.example.com-access.log | awk '{print $7}' | sort -nr | uniq -c | sort -nr | head

To show the bandwidth for a domain use the command below:

# Daily bandwidth total
[[email protected] ~]# grep '27/Feb/2017:' /var/log/httpd/www.example.com-access.log | grep -oP 'HTTP/1.[01]" [0-9]{3} [0-9]+' | awk '{SUM+=$3} END { print SUM / 1024 / 1024 / 1024 " GB" }'

# Monthly bandwidth total
[[email protected] ~]# grep '/Feb/2017:' /var/log/httpd/www.example.com-access.log | grep -oP 'HTTP/1.[01]" [0-9]{3} [0-9]+' | awk '{SUM+=$3} END { print SUM / 1024 / 1024 / 1024 " GB" }'

To get a count of status codes to identify any trends:

# Get all status codes
[[email protected] ~]# cat /var/log/httpd/www.example.com-access.log |awk '{print $9}' | sort -nr | uniq -c |sort -nr
  36355 200
   4896 304
   3942 404
   1599 302
    301 301
    195 403
      4 400
      3 401

# Get summary of top 10 404's:
[[email protected] ~]# awk '($9 ~ /404/)' /var/log/httpd/www.example.com-access.log | awk '{print $9,$7}' | sort -nr | uniq -c |sort -nr | head
   1369 404 /apple-touch-icon-precomposed.png
   1369 404 /apple-touch-icon.png
    502 404 /apple-touch-icon-120x120-precomposed.png
    502 404 /apple-touch-icon-120x120.png
     22 404 /apple-touch-icon-152x152-precomposed.png
     22 404 /apple-touch-icon-152x152.png
     21 404 /news/html
      5 404 /components/com_foxcontact/lib/file-uploader.php
      3 404 /blog/wp-login.php
      1 404 /author/wp-login.php

Basic Apache Hardening

Below are a couple of the more common best practices that should be used when hardening Apache. These are simply some basics for mitigating a few of the more common CVE’s that have been cropping up in Apache.

At the very minimum, disable the trace method and prevent information disclosure by updating the ServerToken and ServerSignature variables. This can be done within Apache by modifying the following file:

# CentOS 5 and 6
vim /etc/httpd/conf.d/security.conf

# Ubuntu 12.04
vim /etc/apache2/conf.d/security

Then set it accordingly as shown below:

# Disable access to the entire file system except for the directories that
# are explicitly allowed later.
#
# This currently breaks the configurations that come with some web application
# Debian packages. It will be made the default for the release after lenny.
#
#<Directory />
#       AllowOverride None
#       Order Deny,Allow
#       Deny from all
#</Directory>


# Changing the following options will not really affect the security of the
# server, but might make attacks slightly more difficult in some cases.

#
# ServerTokens
# This directive configures what you return as the Server HTTP response
# Header. The default is 'Full' which sends information about the OS-Type
# and compiled in modules.
# Set to one of:  Full | OS | Minimal | Minor | Major | Prod
# where Full conveys the most information, and Prod the least.
#
ServerTokens Prod 

#
# Optionally add a line containing the server version and virtual host
# name to server-generated pages (internal error documents, FTP directory
# listings, mod_status and mod_info output etc., but not CGI generated
# documents or custom error documents).
# Set to "EMail" to also include a mailto: link to the ServerAdmin.
# Set to one of:  On | Off | EMail
#
ServerSignature Off 

#
# Allow TRACE method
#
# Set to "extended" to also reflect the request body (only for testing and
# diagnostic purposes).
#
# Set to one of:  On | Off | extended
#
TraceEnable Off

Another common area to lock down further from the vendor defaults is the SSL configuration, which is located in:

# CentOS 5 and 6
vim /etc/httpd/conf.d/ssl.conf

# Ubuntu 12.04
vim /etc/apache2/mods-enabled/ssl.conf

The most common ones I see on security reports are:
Set SSLHonorCipherOrder to ‘on’
Restrict the allowed ciphers in SSLCipherSuite
Enable only secure protocols

The ciphers can be a bit tricky, especially if you have a WAF or IDS in front of your solution. There is not a one size fits all here, so please be sure to test your site after making these changes as they can cause you problems if set incorrectly for your solution. I’ll post some scenarios below.

For securing your ssl.conf against many of the current vulnerabilities posted at the time of this writing, disable TLSv1.0 which will be a requirement come June 2018, and enable forward security, you can use:

SSLCipherSuite EECDH+AESGCM:EECDH+AES256:EECDH+AES128:EDH+AES:RSA+AESGCM:RSA+AES:!ECDSA:!NULL:!MD5:!DSS:!3DES
SSLProtocol -ALL +TLSv1.1 +TLSv1.2
SSLHonorCipherOrder On

If you prefer to leave TLSv1.0 enabled for the time being as you still have clients connecting to your site with unsupported browsers from Windows XP that doesn’t support anything above TLSv1.0, then you can try the following:

SSLCipherSuite ALL:!EXP:!NULL:!ADH:!LOW:!SSLv3:!SSLv2:!MD5:!RC4:!DSS:!3DES
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder On

If you have an Imperva WAF or Alertlogic IDS in front your solution that needs to decrypt the SSL traffic for analysis, so you therefore can’t use forward security since they need to perform a man-in-the-middle on the traffic, but still want to disable insecure ciphers, then modify the variables in the ssl.conf as follows:

SSLCipherSuite HIGH:!MEDIUM:!AESGCM:!ECDH:!aNULL:!ADH:!DH:!EDH:!CAMELLIA:!GCM:!KRB5:!IDEA:!EXP:!eNULL:!LOW:!RC4:!3DES
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder On

As a final note, Mozilla also put out a config generator for this. It can just provide some additional view points of how you can go about the ciphers. The link is here.

Apache Proxypass

Many solutions today are built using highly available configurations that can easily scale. Setting up a solution to scale is easy, but getting your web application to work correctly with a multi-server configuration can be difficult as not everyone has access to a quality shared storage solution that is fast and reliable.

In many web applications such as WordPress, you typically want all your wp-admin traffic to go to the master server. There are probably a dozen ways to go about this, many of which get very over complicated with wacky Varnish configurations handling the redirection, or even with Nginx.

These is where ProxyPass can offer a cleaner alternative. ProxyPass allows you to take a request for a specific URL, and forward it to another server, which would be known as your backend server, or your master web server.

This guide will assume that you are performing this on all web servers in the solution, unless otherwise specified. The specific examples are for a WordPress based solution, but it can be easily adapted for other CMS’s.

To get started, first ensure that mod_proxy is installed:

# CentOS 6
[[email protected] ~]# yum install mod_proxy_html
[[email protected] ~]# service httpd restart
[[email protected] ~]# httpd -M |grep proxy
 proxy_module (shared)
 proxy_balancer_module (shared)
 proxy_ftp_module (shared)
 proxy_http_module (shared)
 proxy_connect_module (shared)
 proxy_ajp_module (shared)

# Ubuntu 12.04 and 14.04
[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install libapache2-mod-proxy-html
[[email protected] ~]# a2enmod proxy proxy_http

There are several ways you can proceed from here. I’ll post them out as ‘options’ below. Each one basically accomplishes the same thing, but one may work better for your environment than another.

So no matter which of the 3 options you go with, always be sure to rigorously test it before implementing it in production!

Option 1: Easy – Define master server based off the URI in each Apache Vhost

This example is simple. In each Apache Vhost, add the following lines on each slave web server to point wp-admin and wp-login.php to your master server, which in this case is 192.168.2.1:

# CentOS 6
[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf

# Ubuntu 12.04 and 14.04
[[email protected] ~]# vim /etc/apache2/sites-enabled/example.com.conf
...
ProxyPreserveHost On
        ProxyRequests Off
        ProxyPassMatch ".*/wp-admin.*" "http://192.168.2.1"
        ProxyPassMatch ".*/wp-login.php" "http://192.168.2.1"
...

Option 2: Advanced – Define master server based off URI using location blocks in each Apache Vhost

This example is slightly more advanced. In each Apache Vhost, add the following location blocks to point wp-admin and wp-login.php to your master server, which in this case is 192.168.2.1. We’re also manually defining the host header within these location blocks, which gives you the option to start excluding specific items if needed:

# CentOS 6
[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf

# Ubuntu 12.04 and 14.04
[[email protected] ~]# vim /etc/apache2/sites-enabled/example.com.conf
...
ProxyRequests Off
  ProxyPreserveHost Off
  ProxyVia Off
  <Location "/wp-login.php">
    Header set "Host" "www.example.com"
    ProxyPass http://192.168.2.1/wp-login.php
    ProxyPassReverse http://192.168.2.1/wp-login.php
  </Location>
  <Location "/wp-admin">
    Header set "Host" "www.example.com"
    ProxyPass http://192.168.2.1/wp-admin
    ProxyPassReverse http://192.168.2.1/wp-admin
  </Location>

Option 3: Complex – Define master server in global Apache configuration, and only send over POST requests for wp-admin

This example is more complex. You are defining the master server (192.168.2.1) in your global Apache configuration, then configuring each Apache Vhost to only send over POST requests for wp-admin to the master server.

Setup proxypass so it knows which server is the master web server. Be sure to update the IP so its the IP address of your master web server:

# CentOS 6
[[email protected] ~]# vim /etc/sysconfig/httpd
...
OPTIONS="-DSLAVE"
export MASTER_SERVER="192.168.2.1"
...

# Ubuntu 12.04 and 14.04
[[email protected] ~]# /etc/apache2/envvars
...
export APACHE_ARGUMENTS="-DSLAVE"
export MASTER_SERVER="192.168.2.1"
...

Now on your slave web servers, we need to update the site’s vhost configuration to proxy the requests for /wp-admin so they will route to the master web server:

# CentOS 6
[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf

# Ubuntu 12.04 and 14.04
[[email protected] ~]# vim /etc/apache2/sites-enabled/example.com.conf
...
<IfDefine SLAVE>
RewriteEngine On
ProxyPreserveHost On
ProxyPass /wp-admin/ http://${MASTER_SERVER}/wp-admin/
     ProxyPassReverse /wp-admin/ http://${MASTER_SERVER}/wp-admin/
RewriteCond %{REQUEST_METHOD} =POST
     RewriteRule . http://${MASTER_SERVER}%{REQUEST_URI} [P]
</IfDefine>
...

# CentOS 6
[[email protected] ~]# service httpd restart

# Ubuntu 12.04 and 14.04
[[email protected] ~]# service apache2 restart

That slave server(s) should now start proxying the /wp-admin requests and sending them over to the master web server. Please be sure to test this out and check your logs to ensure /wp-admin POST requests are now routing to the master web server.

How to install Nginx and PHP-FPM on CentOS

Nginx is a very popular web server in recent years. It is very well known for being used as a high performance web server, a reverse proxy server, or even a load balancer. Some differences Nginx has from Apache:

- No built in PHP support (mod_php).
- Apache is process based while Nginx is event based.
- Modules in Nginx are compiled directly into the binary while Apache has installable modules that can be enabled or disabled quickly
- Nginx configuration files are a series of includes for each sites configuration, whereas Apache has global settings and a companion virtual host with overrides for customization.  So this means there is no override file like Apache's .htaccess in Nginx.
- There are no SSLCertificateChainFile in Nginx.  Intermediate CA certificates need to be appended directly to the site's SSL certificate.

At this time, the current best practice for installing Nginx is to use Nginx’s repos directly as they will contain the most recent and stable releases of Nginx. So install the repo package for Nginx by:

# CentOS 6
[[email protected] ~]# rpm -i http://nginx.org/packages/centos/6/noarch/RPMS/nginx-release-centos-6-0.el6.ngx.noarch.rpm
[[email protected] ~]# rpm --import http://nginx.org/keys/nginx_signing.key
[[email protected] ~]# yum install nginx

# CentOS 7
[[email protected] ~]# rpm -i http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
[[email protected] ~]# rpm --import http://nginx.org/keys/nginx_signing.key
[[email protected] ~]# yum install nginx

As Nginx does not have anything to natively process PHP, install php5-fpm by:

[[email protected] ~]# yum install php-fpm

By default, php-fpm will be listening for connections over TCP. This is incredibly slow when Nginx and php-fpm are on the same server. So modify the php-fpm configuration from listening over TCP to using sockets by:

[[email protected] ~]# vim /etc/php-fpm.d/www.conf
...
[www]
listen = /var/run/php5-fpm.sock
...

Now set php-fpm to run as user nginx:

[[email protected] ~]# vim /etc/php-fpm.d/www.conf
...
user = nginx
group = nginx
listen.owner = nginx
listen.group = nginx
...

Set php-fpm to start on boot, and then startup the service:

# CentOS 6
[[email protected] ~]# chkconfig php-fpm on
[[email protected] ~]# service php-fpm start

# CentOS 7
[[email protected] ~]# systemctl enable php-fpm.service
[[email protected] ~]# systemctl start php-fpm.service

Back in Nginx, configure it to send any PHP requests to the php-fpm socket inside of the http{} block:

[[email protected] ~]# vim /etc/nginx/nginx.conf
http {
...
upstream php5-fpm-sock {
     server unix:/var/run/php5-fpm.sock;
}
...
}

Now that Nginx and php-fpm are configured, setup your first site:

[[email protected] ~]# mkdir -p /var/www/vhosts/example.com
[[email protected] ~]# vim /etc/nginx/conf.d/example.com.conf
server {
     listen 80;
     server_name example.com www.example.com;

     root /var/www/vhosts/example.com;
     index index.php index.html index.htm;

     access_log /var/log/nginx/example.com-access.log;
     error_log /var/log/nginx/example.com-error.log;

     location ~ \.php$ {
          expires off;
          try_files $uri =404;
          include /etc/nginx/fastcgi_params;
          fastcgi_pass php5-fpm-sock;
          fastcgi_index index.php;
          fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
     }

     # caching of files
     location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
          expires max;
          log_not_found off;
     }
}

#server {
#      listen 443 ssl;
#      server_name example.com www.example.com;
#
#      root /var/www/vhosts/example.com;
#      index index.php index.html index.htm;
#
#      access_log /var/log/nginx/example.com-ssl-access.log;
#      error_log /var/log/nginx/example.com-ssl-error.log;
#
#      ssl_certificate /etc/pki/tls/certs/YYYY_example.com.crt;
#      ssl_certificate_key /etc/pki/tls/private/YYYY_example.com.key;
#
#      ssl_session_timeout 5m;
#
#      # Use PCI compliant SSL protocols and ciphers
#      ssl_protocols SSLv3 TLSv1;
#      ssl_ciphers HIGH:!kEDH:!ADH:!EXPORT56;
#      ssl_prefer_server_ciphers on;
#
#      location ~ \.php$ {
#           expires off;
#           try_files $uri =404;
#           include /etc/nginx/fastcgi_params;
#           fastcgi_pass php5-fpm-sock;
#           fastcgi_index index.php;
#           fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
#      }
#
#      # caching of files
#      location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
#           expires max;
#           log_not_found off;
#      }
# }

Finally, start Nginx and enable it to start on boot:

# CentOS 6
[[email protected] ~]# chkconfig nginx on
[[email protected] ~]# service nginx start

# CentOS 7
[[email protected] ~]# systemctl enable nginx.service
[[email protected] ~]# systemctl start nginx.service

NOTE : There is no SSLCertificateChainFile in nginx. CA Certs need to be appended to the certificate for the domain.

Troubleshooting

If you see a page that says ‘The page you are looking for is currently unavailable’, you’ll find Nginx is having trouble getting data from php-fpm (while using a socket). Make sure that the socket you specify in the nginx.conf and the listen line in www.conf pool file for PHP-FPM match.

If you get a ‘File Not Found’ page only on php pages (when using a TCP port for php-fpm) you can check the nginx logs, and you’ll probably see an error indicating it couldn’t connect on the port. Its possible this could be happening if you need to move the root and index definitions out of a location block (if they are defined there). This is because the php section has it’s own location, the documents won’t try to be found using definitions in the domain ‘location /’ block.

How to install Nginx and PHP-FPM on Ubuntu

Nginx is a very popular web server in recent years. It is very well known for being used as a high performance web server, a reverse proxy server, or even a load balancer. Some differences Nginx has from Apache:

- No built in PHP support (mod_php).
- Apache is process based while Nginx is event based.
- Modules in Nginx are compiled directly into the binary while Apache has installable modules that can be enabled or disabled quickly
- Nginx configuration files are a series of includes for each sites configuration, whereas Apache has global settings and a companion virtual host with overrides for customization.  So this means there is no override file like Apache's .htaccess in Nginx.
- There are no SSLCertificateChainFile in Nginx.  Intermediate CA certificates need to be appended directly to the site's SSL certificate.

At this time, the current best practice for installing Nginx is to use Nginx’s repos directly as they will contain the most recent and stable releases of Nginx. So install the repo package for Nginx by:

# Ubuntu 12.04 and 14.04
[[email protected] ~]# nginx=stable # use nginx=development for latest development version
[[email protected] ~]# apt-get install python-software-properties
[[email protected] ~]# add-apt-repository ppa:nginx/$nginx
[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install nginx

As Nginx does not have anything to natively process PHP, install php5-fpm by:

# Ubuntu 12.04 and 14.04
[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install php5-fpm php5-cli

By default, php-fpm will be listening for connections over TCP. This is incredibly slow when Nginx and php-fpm are on the same server. So modify the php-fpm configuration from listening over TCP to using sockets by:

[[email protected] ~]# vim /etc/php5/fpm/pool.d/www.conf
...
[www]
listen = /var/run/php5-fpm.sock
...

Now set php-fpm to run as user nginx:

[[email protected] ~]# vim /etc/php5/fpm/pool.d/www.conf
...
user = www-data
group = www-data
listen.owner = www-data
listen.group = www-data
...

Back in Nginx, configure it to send any PHP requests to the php-fpm socket inside of the http{} block:

[[email protected] ~]# vim /etc/nginx/nginx.conf
http {
...
upstream php5-fpm-sock {
     server unix:/var/run/php5-fpm.sock;
}
...
}

Now that Nginx and php-fpm are configured, setup your first site:

[[email protected] ~]# mkdir -p /var/www/vhosts/example.com
[[email protected] ~]# vim /etc/nginx/conf.d/example.com.conf
server {
     listen 80;
     server_name example.com www.example.com;

     root /var/www/vhosts/example.com;
     index index.php index.html index.htm;

     access_log /var/log/nginx/example.com-access.log;
     error_log /var/log/nginx/example.com-error.log;

     location ~ \.php$ {
          expires off;
          try_files $uri =404;
          include /etc/nginx/fastcgi_params;
          fastcgi_pass php5-fpm-sock;
          fastcgi_index index.php;
          fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
     }

     # caching of files
     location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
          expires max;
          log_not_found off;
     }
}

#server {
#      listen 443 ssl;
#      server_name example.com www.example.com;
#
#      root /var/www/vhosts/example.com;
#      index index.php index.html index.htm;
#
#      access_log /var/log/nginx/example.com-ssl-access.log;
#      error_log /var/log/nginx/example.com-ssl-error.log;
#
#      ssl_certificate /etc/pki/tls/certs/YYYY_example.com.crt;
#      ssl_certificate_key /etc/pki/tls/private/YYYY_example.com.key;
#
#      ssl_session_timeout 5m;
#
#      # Use PCI compliant SSL protocols and ciphers
#      ssl_protocols SSLv3 TLSv1;
#      ssl_ciphers HIGH:!kEDH:!ADH:!EXPORT56;
#      ssl_prefer_server_ciphers on;
#
#      location ~ \.php$ {
#           expires off;
#           try_files $uri =404;
#           include /etc/nginx/fastcgi_params;
#           fastcgi_pass php5-fpm-sock;
#           fastcgi_index index.php;
#           fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
#      }
#
#      # caching of files
#      location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
#           expires max;
#           log_not_found off;
#      }
# }

Finally, restart Nginx and PHP-FPM to apply the changes

# Ubuntu 12.04 and Ubuntu 14.04
[[email protected] ~]# service php5-fpm restart
[[email protected] ~]# service nginx restart

NOTE : There is no SSLCertificateChainFile in nginx. CA Certs need to be appended to the certificate for the domain.

Troubleshooting

If you see a page that says ‘The page you are looking for is currently unavailable’, you’ll find Nginx is having trouble getting data from php-fpm (while using a socket). Make sure that the socket you specify in the nginx.conf and the listen line in www.conf pool file for PHP-FPM match.

If you get a ‘File Not Found’ page only on php pages (when using a TCP port for php-fpm) you can check the nginx logs, and you’ll probably see an error indicating it couldn’t connect on the port. Its possible this could be happening if you need to move the root and index definitions out of a location block (if they are defined there). This is because the php section has it’s own location, the documents won’t try to be found using definitions in the domain ‘location /’ block.

Nginx stub_status module

Like the Apache mod_status module, Nginx also has a module that can provide you with realtime data regarding its stats. While its not as detailed as Apache mod_status, the data is still useful, especially if you are using Nginx as a load balancer.

First, confirm that Nginx was compiled with the HttpStudStatusModule. If you installed this via yum or apt-get, it should have it:

[[email protected] ~]# nginx -V 2>&1 | grep -o with-http_stub_status_module
with-http_stub_status_module

To enable the module, set the following configuration:

[[email protected] ~]# vim /etc/nginx/conf.d/nginx_status.conf
server {

    listen 80 default_server;
    access_log off;
    server_name _;
    server_name_in_redirect off;
    root  /var/www/html;

        location /nginx_status {
                # Enable Nginx stats
                stub_status on;
                # Disable logging for stats
                access_log   off;
                # Security: Only allow access from authorized IP's
                allow 127.0.0.1;
                allow 192.168.1.100;
                # Deny everyone else
                deny all;
        }

}

Now navigate your browser to:

http://serverip/nginx_status

The results will look something like this:

Active connections: 1 
server accepts handled requests
 1 1 1 
Reading: 0 Writing: 1 Waiting: 0

Here is what these numbers mean, taken directly from Nginx’s documentation:

# Active connections
The current number of active client connections including Waiting connections.

# accepts
The total number of accepted client connections.

# handled
The total number of handled connections. Generally, the parameter value is the same as accepts unless some resource limits have been reached (for example, the worker_connections limit).

# requests
The total number of client requests.

# Reading
The current number of connections where nginx is reading the request header.

# Writing
The current number of connections where nginx is writing the response back to the client.

# Waiting
The current number of idle client connections waiting for a request.