How to install and configure Redis on Ubuntu 16.04

Redis is an in-memory data structure store that is commonly used as a database, cache or a message broker. What makes Redis powerful is its optional ability to have data persistence. Therefore your dataset isn’t lost during restarts of the service.

The article below will discuss how to install, configure and provide basic security for Redis. From there, it will go into the basic use cases I use it for on a daily basis, session storing and general caching.

Installation

[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install redis-server redis-tools

Configuration

Redis listens on port 6379 by default and needs some additional configuration to ensure that it is secured. If you do not protect Redis with a firewall, authentication and have it listen only on a private network, there is a extremely high risk of leaking sensitive data.

First, set Redis to only listen on your private network. Redis does not have any type of encryption built in, it is important that the data is transferred only over private networks or secured tunnels. Set Redis to listen on the private interface by:

[[email protected] ~]# vim /etc/redis/redis.conf
...
bind redis_servers_private_IP
...

If Redis is being installed on a stand alone web server and will not need to accept connections from other clients, then you can set Redis to listen on the local socket instead by commenting out the bind value and setting up a socket by:

[[email protected] ~]# mkdir /var/run/redis
[[email protected] ~]# chown redis:redis /var/run/redis
[[email protected] ~]# vim /etc/redis/redis.conf
...
# bind 127.0.0.1
unixsocket /var/run/redis/redis.sock
unixsocketperm 777

If you do not have a dedicated firewall, use your OS’s built in firewall to only allow in connections from trusted web servers using their internal IP’s. Some quick examples are below:

# iptables
[[email protected] ~]# vim /etc/sysconfig/iptables
...
-A INPUT -p tcp -m tcp --dport 6379 -s client_server_private_IP -m comment --comment "redis" -j ACCEPT
[[email protected] ~]# service iptables restart

# ufw
[[email protected] ~]# ufw allow from client_server_private_IP/32 to any port 6379

To protect Redis further, setup authentication which is a built in security feature. This will force clients to authenticate before being granted access. Use a tool such as apg or pwgen to create a secure password. Set the password within Redis by:

[[email protected] ~]# vim /etc/redis/redis.conf
...
requirepass your_secure_password_here
...

[[email protected] ~]# systemctl restart redis

Then test to ensure the password works by:

# This should fail
[[email protected] ~]# redis_cli
127.0.0.1:6379> set key1 10
(error) NOAUTH Authentication required.

# This should work
[[email protected] ~]# redis-cli
127.0.0.1:6379> auth your_secure_password_here
127.0.0.1:6379> set key1 10
OK
127.0.0.1:6379> get key1
"10"

Next, we need to secure the file permissions for Redis. The redis.conf contains the password for redis, so that file shouldn’t be readable by everyone. We also want to lock down the Redis data directory. Lock down the permissions on Redis by:

[[email protected] ~]# chmod 700 /var/lib/redis
[[email protected] ~]# chown redis:redis /etc/redis/redis.conf
[[email protected] ~]# chmod 600 /etc/redis/redis.conf
[[email protected] ~]# systemctl restart redis

The Official Redis Administration Guide recommends to disable Transparent Huge Pages (THP). This can be performed live by:

[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

And disable Transparent Huge Pages (THP) at boot time by making a systemd unit file:

[[email protected] ~]# vim /etc/systemd/system/disable-thp.service
[Unit]
Description=Disable Transparent Huge Pages (THP)

[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl start disable-thp
[[email protected] ~]# systemctl enable disable-thp

The Official Redis Administration Guide also recommends setting the following sysctl:

[[email protected] ~]# sysctl vm.overcommit_memory=1
[[email protected] ~]# vim /etc/sysctl.conf
...
vm.overcommit_memory = 1
...

Redis Configurations

Now that Redis is installed and secured, its time to tune it for your application’s needs. There are typically 2 types of Redis cache configurations I see:
Session store
Database cache or full page cache

The session cache allows for a single locations for your application to store sessions that would normally be stored by PHP on the local file system. It is important to ensure that the data is saved to disk so you don’t lose all the sessions between restarts of Redis. This is one of the primary advantages of using Redis over Memcached.

The full page cache is great for caching SQL queries or for serving as a full page cache for applications such as Magento or WordPress, caching images, videos, html, css or js. Generally this type of cache doesn’t need to be persistent across restarts of Redis.

Redis Configurations – Session store

When using Redis as a session store, you want to ensure that the Redis data persists between restarts of Redis. Otherwise your users could be left wondering why their shopping carts all of a sudden vanished. The example below will have disk persistence enabled with a memory limit of 1.5G (1536M).

These settings may or may not work for you! Adjust your settings to meet your environments requirements!

[[email protected] ~]# vim /etc/redis/redis.conf
...
## Ensure disk persistence is enabled
save 900 1
save 300 10
save 60 10000
...
## Set the max memory
maxmemory 1536mb
...

[[email protected] ~]# service redis restart

Redis Configurations – Database cache or full page cache

When using Redis for database caching or as a FPC for applications like WordPress or Magento, I disable disk persistence. This means the cache will only be stored in memory and lost whenever redis restart. I also set the memory limit to 1.5G (1536M) for starters and adjust accordingly from there. Since I am only storing cached data, I can avoid out of memory issues by allowing Redis to automatically remove the oldest cache entries using the maxmemory-policy allkeys-lru. Read up on the Redis supported eviction policies here.

These settings may or may not work for you! Adjust your settings to meet your environments requirements! Remember, this example assumes everything in Redis can be lost when the service restarts and the eviction policy will remove the least used keys out of all the data. I typically find this works for Magento and WordPress redis setups:

[[email protected] ~]# vim /etc/redis/redis.conf
...
## Disable disk persistence
#save 900 1
#save 300 10
#save 60 10000
...
## Set the max memory
maxmemory 1536mb
...
## Update the eviction policy
maxmemory-policy allkeys-lru
...

[[email protected] ~]# systemctl restart redis

Multiple Redis Configurations

Redis has the ability to utilize multiple caching configurations with their own settings. The only requirement is that each instance of Redis listens on a unique port, has a unique pid, and of course has its own config and startup script. In the example below, we are going to have 2 redis instances running called redis-sessions and redis-cache. To avoid confusion, we will disable the original redis instance.

# Create 2 new configs and modify the values accordingly
[[email protected] ~]# cp /etc/redis/redis.conf /etc/redis/redis-session.conf
[[email protected] ~]# vim /etc/redis/redis-session.conf
...
pidfile /var/run/redis-session.pid
port 6379
logfile /var/log/redis/redis-session.log
dir /var/lib/redis-session
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-session.sock
unixsocketperm 777
...

[[email protected] ~]# cp /etc/redis/redis.conf /etc/redis/redis-cache.conf
[[email protected] ~]# vim /etc/redis/redis-cache.conf
...
pidfile /var/run/redis-cache.pid
port 6380
logfile /var/log/redis/redis-cache.log
dir /var/lib/redis-cache
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-cache.sock
unixsocketperm 777
...

# Create directories and secure the permissions
[[email protected] ~]# mkdir /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chown redis:redis /var/lib/redis-session /var/lib/redis-cache /etc/redis/redis-session.conf /etc/redis/redis-cache.conf
[[email protected] ~]# chmod 700 /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chmod 600 /etc/redis/redis-session.conf /etc/redis/redis-cache.conf
[[email protected] ~]# cd /etc/redis
[[email protected] ~]# cp -Rp redis-server.post-down.d redis-session.post-down.d && cp -Rp redis-server.post-up.d redis-session.post-up.d && cp -Rp redis-server.pre-down.d redis-session.pre-down.d && cp -Rp redis-server.pre-up.d  redis-session.pre-up.d 
[[email protected] ~]# cp -Rp redis-server.post-down.d redis-cache.post-down.d && cp -Rp redis-server.post-up.d redis-cache.post-up.d && cp -Rp redis-server.pre-down.d redis-cache.pre-down.d && cp -Rp redis-server.pre-up.d  redis-cache.pre-up.d 

# Create startup files
[[email protected] ~]# cp /etc/systemd/system/redis.service /etc/systemd/system/redis-session.service
[[email protected] ~]# vim /etc/systemd/system/redis-session.service
...
ExecStart=/usr/bin/redis-server /etc/redis/redis-session.conf
PIDFile=/var/run/redis/redis-session.pid
ExecStartPre=-/bin/run-parts --verbose /etc/redis/redis-session.pre-up.d
ExecStartPost=-/bin/run-parts --verbose /etc/redis/redis-session.post-up.d
ExecStop=-/bin/run-parts --verbose /etc/redis/redis-session.pre-down.d
ExecStopPost=-/bin/run-parts --verbose /etc/redis/redis-session.post-down.d
ReadWriteDirectories=-/var/lib/redis-session
Alias=redis-session.service
...
[[email protected] ~]# cp /etc/systemd/system/redis.service /etc/systemd/system/redis-cache.service
[[email protected] ~]# vim /etc/systemd/system/redis-cache.service
...
ExecStart=/usr/bin/redis-server /etc/redis/redis-cache.conf
PIDFile=/var/run/redis/redis-cache.pid
ExecStartPre=-/bin/run-parts --verbose /etc/redis/redis-cache.pre-up.d
ExecStartPost=-/bin/run-parts --verbose /etc/redis/redis-cache.post-up.d
ExecStop=-/bin/run-parts --verbose /etc/redis/redis-cache.pre-down.d
ExecStopPost=-/bin/run-parts --verbose /etc/redis/redis-cache.post-down.d
ReadWriteDirectories=-/var/lib/redis-cache
Alias=redis-cache.service
...

# Stop and disable old instance, start new instances.  Error during 'systemctl enable' appears safe to ignore as service still starts on boot.  But confirm for yourself!
[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl stop redis.service && systemctl disable redis.service
[[email protected] ~]# systemctl enable redis-session.service && systemctl start redis-session.service
[[email protected] ~]# systemctl enable redis-cache.service && systemctl start redis-cache.service

# Finally, edit the /etc/redis/redis-session.conf and /etc/redis/redis-cache.conf using the instructions earlier in this article for configuring sessions and db cache.

Client setup

The typical use cases I run into on a day to day basis are clients using Redis for their PHP application. Redis can be used to store cached content, or it can be used to centrally store sessions. Therefore these examples will be PHP focused.

Client setup – General data caching

For storing data, there is nothing that needs to be configured on the client side. The application code itself is what controls storing content within Redis.

Client setup – Storing sessions in Redis

To have Redis act as a central server for sessions, some additional configuration is needed on each client web server. Install php-memcache for your version of PHP. Assuming the default PHP version is installed from the package manager, you can install it by:

[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install php-redis redis-tools
[[email protected] ~]# service apache2 graceful
[[email protected] ~]# php -m |grep redis

Then update the php.ini as follows:

session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=your_secure_password_here"

Test to ensure sessions are now being stored in Redis:

[[email protected] ~]# vim /var/www/html/test-sessions.php
<?php
session_start();
?>
Created a session

Then run the following on the command line and confirm the returned numbers increment as shown below:

[[email protected] ~]# curl localhost/test-sessions.php && redis-cli -a your_secure_password_here keys '*' |grep SESSION | wc -l
10
Created a session
11

Troubleshooting

Confirm Redis is online:

[[email protected] ~]# redis-cli ping

How to connect using redis-cli when redis is running on a different server or using a different port:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here

Sometimes you may need to flush the Redis cache. Before doing this, make sure you are connecting to the right instance of Redis since there could be multiple instances running. An example is below:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here
FLUSHALL

To get some useful stats on Redis, run:

[[email protected] ~]# redis-cli
info

To get memory specific stats, run:

[[email protected] ~]# redis-cli
info memory
127.0.0.1:6379> info memory
# Memory
used_memory:488315720
used_memory_human:465.69M
used_memory_rss:499490816
used_memory_peak:505227288
used_memory_peak_human:481.82M
used_memory_lua:36864
mem_fragmentation_ratio:1.02
mem_allocator:jemalloc-3.6.0

To increase the memory limit assigned to Redis without restarting the service, the example shown below will increase the memory allocated dynamically from 1G to 2G without a restart of the Redis:

[[email protected] ~]# redis-cli
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "1000000000"
127.0.0.1:6379> config set maxmemory 2g
OK
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "2000000000"

Regarding performance issues with Redis, there are a number of factors that need to be account for, too many for this article. Redis published an excellent article that goes into various things that could cause latency with Redis at:
https://redis.io/topics/latency

How to install and configure Redis on Ubuntu 14.04

Redis is an in-memory data structure store that is commonly used as a database, cache or a message broker. What makes Redis powerful is its optional ability to have data persistence. Therefore your dataset isn’t lost during restarts of the service.

The article below will discuss how to install, configure and provide basic security for Redis. From there, it will go into the basic use cases I use it for on a daily basis, session storing and general caching.

Installation

[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install redis-server redis-tools

Configuration

Redis listens on port 6379 by default and needs some additional configuration to ensure that it is secured. If you do not protect Redis with a firewall, authentication and have it listen only on a private network, there is a extremely high risk of leaking sensitive data.

First, set Redis to only listen on your private network. Redis does not have any type of encryption built in, it is important that the data is transferred only over private networks or secured tunnels. Set Redis to listen on the private interface by:

[[email protected] ~]# vim /etc/redis/redis.conf
...
bind redis_servers_private_IP
...

If Redis is being installed on a stand alone web server and will not need to accept connections from other clients, then you can set Redis to listen on the local socket instead by commenting out the bind value and setting up a socket by:

[[email protected] ~]# mkdir /var/run/redis
[[email protected] ~]# chown redis:redis /var/run/redis
[[email protected] ~]# vim /etc/redis/redis.conf
...
# bind 127.0.0.1
unixsocket /var/run/redis/redis.sock
unixsocketperm 777

If you do not have a dedicated firewall, use your OS’s built in firewall to only allow in connections from trusted web servers using their internal IP’s. Some quick examples are below:

# iptables
[[email protected] ~]# vim /etc/sysconfig/iptables
...
-A INPUT -p tcp -m tcp --dport 6379 -s client_server_private_IP -m comment --comment "redis" -j ACCEPT
[[email protected] ~]# service iptables restart

# ufw
[[email protected] ~]# ufw allow from client_server_private_IP/32 to any port 6379

To protect Redis further, setup authentication which is a built in security feature. This will force clients to authenticate before being granted access. Use a tool such as apg or pwgen to create a secure password. Set the password within Redis by:

[[email protected] ~]# vim /etc/redis/redis.conf
...
requirepass your_secure_password_here
...

[[email protected] ~]# service redis restart

Then test to ensure the password works by:

# This should fail
[[email protected] ~]# redis_cli
127.0.0.1:6379> set key1 10
(error) NOAUTH Authentication required.

# This should work
[[email protected] ~]# redis-cli
127.0.0.1:6379> auth your_secure_password_here
127.0.0.1:6379> set key1 10
OK
127.0.0.1:6379> get key1
"10"

Next, we need to secure the file permissions for Redis. The redis.conf contains the password for redis, so that file shouldn’t be readable by everyone. We also want to lock down the Redis data directory. Lock down the permissions on Redis by:

[[email protected] ~]# chmod 700 /var/lib/redis
[[email protected] ~]# chown redis:redis /etc/redis/redis.conf
[[email protected] ~]# chmod 600 /etc/redis/redis.conf
[[email protected] ~]# service redis restart

The Official Redis Administration Guide recommends to disable Transparent Huge Pages (THP). This can be performed live by:

[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

And disable Transparent Huge Pages (THP) at boot time by entering the following before the ‘exit0’:

[[email protected] ~]# vim /etc/rc.local
...
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
...

The Official Redis Administration Guide also recommends setting the following sysctl:

[[email protected] ~]# sysctl vm.overcommit_memory=1
[[email protected] ~]# vim /etc/sysctl.conf
...
vm.overcommit_memory = 1
...

Redis Configurations

Now that Redis is installed and secured, its time to tune it for your application’s needs. There are typically 2 types of Redis cache configurations I see:
Session store
Database cache or full page cache

The session cache allows for a single locations for your application to store sessions that would normally be stored by PHP on the local file system. It is important to ensure that the data is saved to disk so you don’t lose all the sessions between restarts of Redis. This is one of the primary advantages of using Redis over Memcached.

The full page cache is great for caching SQL queries or for serving as a full page cache for applications such as Magento or WordPress, caching images, videos, html, css or js. Generally this type of cache doesn’t need to be persistent across restarts of Redis.

Redis Configurations – Session store

When using Redis as a session store, you want to ensure that the Redis data persists between restarts of Redis. Otherwise your users could be left wondering why their shopping carts all of a sudden vanished. The example below will have disk persistence enabled with a memory limit of 1.5G (1536M).

These settings may or may not work for you! Adjust your settings to meet your environments requirements!

[[email protected] ~]# vim /etc/redis/redis.conf
...
## Ensure disk persistence is enabled
save 900 1
save 300 10
save 60 10000
...
## Set the max memory
maxmemory 1536mb
...

[[email protected] ~]# service redis restart

Redis Configurations – Database cache or full page cache

When using Redis for database caching or as a FPC for applications like WordPress or Magento, I disable disk persistence. This means the cache will only be stored in memory and lost whenever redis restart. I also set the memory limit to 1.5G (1536M) for starters and adjust accordingly from there. Since I am only storing cached data, I can avoid out of memory issues by allowing Redis to automatically remove the oldest cache entries using the maxmemory-policy allkeys-lru. Read up on the Redis supported eviction policies here.

These settings may or may not work for you! Adjust your settings to meet your environments requirements! Remember, this example assumes everything in Redis can be lost when the service restarts and the eviction policy will remove the least used keys out of all the data. I typically find this works for Magento and WordPress redis setups:

[[email protected] ~]# vim /etc/redis/redis.conf
...
## Disable disk persistence
#save 900 1
#save 300 10
#save 60 10000
...
## Set the max memory
maxmemory 1536mb
...
## Update the eviction policy
maxmemory-policy allkeys-lru
...

[[email protected] ~]# service redis restart

Multiple Redis Configurations

Redis has the ability to utilize multiple caching configurations with their own settings. The only requirement is that each instance of Redis listens on a unique port, has a unique pid, and of course has its own config and startup script. In the example below, we are going to have 2 redis instances running called redis-sessions and redis-cache. To avoid confusion, we will disable the original redis instance.

# Create 2 new configs and modify the values accordingly
[[email protected] ~]# cp /etc/redis/redis.conf /etc/redis/redis-session.conf
[[email protected] ~]# vim /etc/redis/redis-session.conf
...
pidfile /var/run/redis/redis_session.pid
port 6379
logfile /var/log/redis/redis-session.log
dir /var/lib/redis-session
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-session.sock
unixsocketperm 777
...

[[email protected] ~]# cp /etc/redis/redis.conf /etc/redis/redis-cache.conf
[[email protected] ~]# vim /etc/redis/redis-cache.conf
...
pidfile /var/run/redis_cache.pid
port 6380
logfile /var/log/redis/redis-cache.log
dir /var/lib/redis-cache
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-cache.sock
unixsocketperm 777
...

# Create directories and secure the permissions
[[email protected] ~]# mkdir /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chown redis:redis /var/lib/redis-session /var/lib/redis-cache /etc/redis/redis-session.conf /etc/redis/redis-cache.conf
[[email protected] ~]# chmod 700 /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chmod 600 /etc/redis/redis-session.conf /etc/redis/redis-cache.conf

# Create startup files
[[email protected] ~]# cp /etc/init.d/redis-server /etc/init.d/redis-session
[[email protected] ~]# vim /etc/init.d/redis-session
...
DAEMON_ARGS=/etc/redis/redis-session.conf
PIDFILE=$RUNDIR/redis-session.pid
...
[[email protected] ~]# cp /etc/init.d/redis-server /etc/init.d/redis-cache
[[email protected] ~]# vim /etc/init.d/redis-cache
...
DAEMON_ARGS=/etc/redis/redis-cache.conf
PIDFILE=$RUNDIR/redis-cache.pid
...

# Stop and disable old instance, start new instances
[[email protected] ~]# service redis-server stop && update-rc.d redis-server disable
[[email protected] ~]# service redis-session start && update-rc.d redis-session defaults
[[email protected] ~]# service redis-cache start && update-rc.d redis-cache defaults

# Finally, edit the /etc/redis/redis-session.conf and /etc/redis/redis-cache.conf using the instructions earlier in this article for configuring sessions and db cache.

Client setup

The typical use cases I run into on a day to day basis are clients using Redis for their PHP application. Redis can be used to store cached content, or it can be used to centrally store sessions. Therefore these examples will be PHP focused.

Client setup – General data caching

For storing data, there is nothing that needs to be configured on the client side. The application code itself is what controls storing content within Redis.

Client setup – Storing sessions in Redis

To have Redis act as a central server for sessions, some additional configuration is needed on each client web server. Install php-memcache for your version of PHP. Assuming the default PHP version is installed from the package manager, you can install it by:

[[email protected] ~]# apt-get update
[[email protected] ~]# apt-get install php-redis redis-tools
[[email protected] ~]# service apache2 graceful
[[email protected] ~]# php -m |grep redis

Then update the php.ini as follows:

session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=your_secure_password_here"

Test to ensure sessions are now being stored in Redis:

[[email protected] ~]# vim /var/www/html/test-sessions.php
<?php
session_start();
?>
Created a session

Then run the following on the command line and confirm the returned numbers increment as shown below:

[[email protected] ~]# curl localhost/test-sessions.php && redis-cli -a your_secure_password_here keys '*' |grep SESSION | wc -l
10
Created a session
11

Troubleshooting

Confirm Redis is online:

[[email protected] ~]# redis-cli ping

How to connect using redis-cli when redis is running on a different server or using a different port:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here

Sometimes you may need to flush the Redis cache. Before doing this, make sure you are connecting to the right instance of Redis since there could be multiple instances running. An example is below:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here
FLUSHALL

To get some useful stats on Redis, run:

[[email protected] ~]# redis-cli
info

To get memory specific stats, run:

[[email protected] ~]# redis-cli
info memory
127.0.0.1:6379> info memory
# Memory
used_memory:488315720
used_memory_human:465.69M
used_memory_rss:499490816
used_memory_peak:505227288
used_memory_peak_human:481.82M
used_memory_lua:36864
mem_fragmentation_ratio:1.02
mem_allocator:jemalloc-3.6.0

To increase the memory limit assigned to Redis without restarting the service, the example shown below will increase the memory allocated dynamically from 1G to 2G without a restart of the Redis:

[[email protected] ~]# redis-cli
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "1000000000"
127.0.0.1:6379> config set maxmemory 2g
OK
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "2000000000"

Regarding performance issues with Redis, there are a number of factors that need to be account for, too many for this article. Redis published an excellent article that goes into various things that could cause latency with Redis at:
https://redis.io/topics/latency

How to install and configure Redis on CentOS 7

Redis is an in-memory data structure store that is commonly used as a database, cache or a message broker. What makes Redis powerful is its optional ability to have data persistence. Therefore your dataset isn’t lost during restarts of the service.

The article below will discuss how to install, configure and provide basic security for Redis. From there, it will go into the basic use cases I use it for on a daily basis, session storing and general caching.

Installation

[[email protected] ~]# yum install epel-release
[[email protected] ~]# yum install redis
[[email protected] ~]# systemctl enable redis
[[email protected] ~]# systemctl start redis.service
[[email protected] ~]# redis-cli ping

Configuration

Redis listens on port 6379 by default and needs some additional configuration to ensure that it is secured. If you do not protect Redis with a firewall, authentication and have it listen only on a private network, there is a extremely high risk of leaking sensitive data.

First, set Redis to only listen on your private network. Redis does not have any type of encryption built in, it is important that the data is transferred only over private networks or secured tunnels. Set Redis to listen on the private interface by:

[[email protected] ~]# vim /etc/redis.conf
...
bind redis_servers_private_IP
...

If Redis is being installed on a stand alone web server and will not need to accept connections from other clients, then you can set Redis to listen on the local socket instead by commenting out the bind value and setting up a socket by:

[[email protected] ~]# mkdir /var/run/redis
[[email protected] ~]# chown redis:redis /var/run/redis
[[email protected] ~]# vim /etc/redis.conf
...
# bind 127.0.0.1
unixsocket /var/run/redis/redis.sock
unixsocketperm 777

If you do not have a dedicated firewall, use your OS’s built in firewall to only allow in connections from trusted web servers using their internal IP’s. Some quick examples are below:

# iptables
[[email protected] ~]# vim /etc/sysconfig/iptables
...
-A INPUT -p tcp -m tcp --dport 6379 -s client_server_private_IP -m comment --comment "redis" -j ACCEPT
[[email protected] ~]# service iptables restart

# firewalld
[[email protected] ~]# firewall-cmd --permanent --new-zone=redis
[[email protected] ~]# firewall-cmd --permanent --zone=redis --add-port=6379/tcp
[[email protected] ~]# firewall-cmd --permanent --zone=redis --add-source=client_server_private_IP
[[email protected] ~]# firewall-cmd --reload

To protect Redis further, setup authentication which is a built in security feature. This will force clients to authenticate before being granted access. Use a tool such as apg or pwgen to create a secure password. Set the password within Redis by:

[[email protected] ~]# vim /etc/redis.conf
...
requirepass your_secure_password_here
...

[[email protected] ~]# systemctl restart redis

Then test to ensure the password works by:

# This should fail
[[email protected] ~]# redis_cli
127.0.0.1:6379> set key1 10
(error) NOAUTH Authentication required.

# This should work
[[email protected] ~]# redis-cli
127.0.0.1:6379> auth your_secure_password_here
127.0.0.1:6379> set key1 10
OK
127.0.0.1:6379> get key1
"10"

Next, we need to secure the file permissions for Redis. The redis.conf contains the password for redis, so that file shouldn’t be readable by everyone. We also want to lock down the Redis data directory. Lock down the permissions on Redis by:

[[email protected] ~]# chown redis:redis /var/lib/redis
[[email protected] ~]# chmod 700 /var/lib/redis
[[email protected] ~]# chown redis:redis /etc/redis.conf
[[email protected] ~]# chmod 600 /etc/redis.conf
[[email protected] ~]# systemctl restart redis

The Official Redis Administration Guide recommends to disable Transparent Huge Pages (THP). This can be performed live by:

[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

And disable Transparent Huge Pages (THP) at boot time by making a systemd unit file:

[[email protected] ~]# vim /etc/systemd/system/disable-thp.service
[Unit]
Description=Disable Transparent Huge Pages (THP)

[Service]
Type=simple
ExecStart=/bin/sh -c "echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled && echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag"

[Install]
WantedBy=multi-user.target

[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl start disable-thp
[[email protected] ~]# systemctl enable disable-thp

The Official Redis Administration Guide also recommends setting the following sysctl:

[[email protected] ~]# sysctl vm.overcommit_memory=1
[[email protected] ~]# vim /etc/sysctl.conf
...
vm.overcommit_memory = 1
...

Redis Configurations

Now that Redis is installed and secured, its time to tune it for your application’s needs. There are typically 2 types of Redis cache configurations I see:
Session store
Database cache or full page cache

The session cache allows for a single locations for your application to store sessions that would normally be stored by PHP on the local file system. It is important to ensure that the data is saved to disk so you don’t lose all the sessions between restarts of Redis. This is one of the primary advantages of using Redis over Memcached.

The full page cache is great for caching SQL queries or for serving as a full page cache for applications such as Magento or WordPress, caching images, videos, html, css or js. Generally this type of cache doesn’t need to be persistent across restarts of Redis.

Redis Configurations – Session store

When using Redis as a session store, you want to ensure that the Redis data persists between restarts of Redis. Otherwise your users could be left wondering why their shopping carts all of a sudden vanished. The example below will have disk persistence enabled with a memory limit of 1.5G (1536M).

These settings may or may not work for you! Adjust your settings to meet your environments requirements!

[[email protected] ~]# vim /etc/redis.conf
...
## Ensure disk persistence is enabled
save 900 1
save 300 10
save 60 10000
...
## Set the max memory
maxmemory 1536mb
...

[[email protected] ~]# systemctl restart redis

Redis Configurations – Database cache or full page cache

When using Redis for database caching or as a FPC for applications like WordPress or Magento, I disable disk persistence. This means the cache will only be stored in memory and lost whenever redis restart. I also set the memory limit to 1.5G (1536M) for starters and adjust accordingly from there. Since I am only storing cached data, I can avoid out of memory issues by allowing Redis to automatically remove the oldest cache entries using the maxmemory-policy allkeys-lru. Read up on the Redis supported eviction policies here.

These settings may or may not work for you! Adjust your settings to meet your environments requirements! Remember, this example assumes everything in Redis can be lost when the service restarts and the eviction policy will remove the least used keys out of all the data. I typically find this works for Magento and WordPress redis setups:

[[email protected] ~]# vim /etc/redis.conf
...
## Disable disk persistence
#save 900 1
#save 300 10
#save 60 10000
...
## Set the max memory
maxmemory 1536mb
...
## Update the eviction policy
maxmemory-policy allkeys-lru
...

[[email protected] ~]# systemctl restart redis

Multiple Redis Configurations

Redis has the ability to utilize multiple caching configurations with their own settings. The only requirement is that each instance of Redis listens on a unique port, has a unique pid, and of course has its own config and startup script. In the example below, we are going to have 2 redis instances running called redis-sessions and redis-cache. To avoid confusion, we will disable the original redis instance.

# Create 2 new configs and modify the values accordingly
[[email protected] ~]# cp /etc/redis.conf /etc/redis-session.conf
[[email protected] ~]# vim /etc/redis-session.conf
...
pidfile /var/run/redis_session.pid
port 6379
logfile /var/log/redis/redis-session.log
dir /var/lib/redis-session
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-session.sock
unixsocketperm 777
...

[[email protected] ~]# cp /etc/redis.conf /etc/redis-cache.conf
...
pidfile /var/run/redis_cache.pid
port 6380
logfile /var/log/redis/redis-cache.log
dir /var/lib/redis-cache
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-cache.sock
unixsocketperm 777
...

# Create directories and secure the permissions
[[email protected] ~]# mkdir /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chown redis:redis /var/lib/redis-session /var/lib/redis-cache /etc/redis-session.conf /etc/redis-cache.conf
[[email protected] ~]# chmod 700 /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chmod 600 /etc/redis-session.conf /etc/redis-cache.conf

# Create startup files
[[email protected] ~]# cp /usr/lib/systemd/system/redis.service /usr/lib/systemd/system/redis-session.service
[[email protected] ~]# vim /usr/lib/systemd/system/redis-session.service
...
ExecStart=/usr/bin/redis-server /etc/redis-session.conf --daemonize no
...
[[email protected] ~]# cp /usr/lib/systemd/system/redis.service /usr/lib/systemd/system/redis-cache.service
[[email protected] ~]# vim /usr/lib/systemd/system/redis-cache.service
...
ExecStart=/usr/bin/redis-server /etc/redis-cache.conf --daemonize no
...

# Stop and disable old instance, start new instances
[[email protected] ~]# systemctl daemon-reload
[[email protected] ~]# systemctl stop redis.service && systemctl disable redis.service
[[email protected] ~]# systemctl enable redis-session.service && systemctl start redis-session.service
[[email protected] ~]# systemctl enable redis-cache.service && systemctl start redis-cache.service

# Finally, edit the /etc/redis-session.conf and /etc/redis-cache.conf using the instructions earlier in this article for configuring sessions and db cache.

Client setup

The typical use cases I run into on a day to day basis are clients using Redis for their PHP application. Redis can be used to store cached content, or it can be used to centrally store sessions. Therefore these examples will be PHP focused.

Client setup – General data caching

For storing data, there is nothing that needs to be configured on the client side. The application code itself is what controls storing content within Redis.

Client setup – Storing sessions in Redis

To have Redis act as a central server for sessions, some additional configuration is needed on each client web server. Install php-memcache for your version of PHP. Assuming the default PHP version is installed from the package manager, you can install it by:

[[email protected] ~]# yum install php-pecl-redis
[[email protected] ~]# service httpd graceful
[[email protected] ~]# php -m |grep redis

Then update the php.ini as follows:

session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=your_secure_password_here"

On CentOS and Red Hat servers, depending on what version of PHP was installed and how, you may have to update another file as it will override the php.ini. Only change this if the values exist and are configured for files:

[[email protected] ~]# vim /etc/httpd/conf.d/php.conf
session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=your_secure_password_here"

Test to ensure sessions are now being stored in Redis:

[[email protected] ~]# vim /var/www/html/test-sessions.php
<?php
session_start();
?>
Created a session

Then run the following on the command line and confirm the returned numbers increment as shown below:

[[email protected] ~]# curl localhost/test-sessions.php && redis-cli -a your_secure_password_here keys '*' |grep SESSION | wc -l
10
Created a session
11

Troubleshooting

Confirm Redis is online:

[[email protected] ~]# redis-cli ping

How to connect using redis-cli when redis is running on a different server or using a different port:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here

Sometimes you may need to flush the Redis cache. Before doing this, make sure you are connecting to the right instance of Redis since there could be multiple instances running. An example is below:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here
FLUSHALL

To get some useful stats on Redis, run:

[[email protected] ~]# redis-cli
info

To get memory specific stats, run:

[[email protected] ~]# redis-cli
info memory
127.0.0.1:6379> info memory
# Memory
used_memory:488315720
used_memory_human:465.69M
used_memory_rss:499490816
used_memory_peak:505227288
used_memory_peak_human:481.82M
used_memory_lua:36864
mem_fragmentation_ratio:1.02
mem_allocator:jemalloc-3.6.0

To increase the memory limit assigned to Redis without restarting the service, the example shown below will increase the memory allocated dynamically from 1G to 2G without a restart of the Redis:

[[email protected] ~]# redis-cli
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "1000000000"
127.0.0.1:6379> config set maxmemory 2g
OK
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "2000000000"

Regarding performance issues with Redis, there are a number of factors that need to be account for, too many for this article. Redis published an excellent article that goes into various things that could cause latency with Redis at:
https://redis.io/topics/latency

How to install and configure Redis on CentOS 6

Redis is an in-memory data structure store that is commonly used as a database, cache or a message broker. What makes Redis powerful is its optional ability to have data persistence. Therefore your dataset isn’t lost during restarts of the service.

The article below will discuss how to install, configure and provide basic security for Redis. From there, it will go into the basic use cases I use it for on a daily basis, session storing and general caching.

Installation

[[email protected] ~]# yum install epel-release
[[email protected] ~]# yum install redis
[[email protected] ~]# chkconfig redis on
[[email protected] ~]# service redis start
[[email protected] ~]# redis-cli ping

Configuration

Redis listens on port 6379 by default and needs some additional configuration to ensure that it is secured. If you do not protect Redis with a firewall, authentication and have it listen only on a private network, there is a extremely high risk of leaking sensitive data.

First, set Redis to only listen on your private network. Redis does not have any type of encryption built in, it is important that the data is transferred only over private networks or secured tunnels. Set Redis to listen on the private interface by:

[[email protected] ~]# vim /etc/redis.conf
...
bind redis_servers_private_IP
...

If Redis is being installed on a stand alone web server and will not need to accept connections from other clients, then you can set Redis to listen on the local socket instead by commenting out the bind value and setting up a socket by:

[[email protected] ~]# mkdir /var/run/redis
[[email protected] ~]# chown redis:redis /var/run/redis
[[email protected] ~]# vim /etc/redis.conf
...
# bind 127.0.0.1
unixsocket /var/run/redis/redis.sock
unixsocketperm 777

If you do not have a dedicated firewall, use your OS’s built in firewall to only allow in connections from trusted web servers using their internal IP’s. Some quick examples are below:

# iptables
[[email protected] ~]# vim /etc/sysconfig/iptables
...
-A INPUT -p tcp -m tcp --dport 6379 -s client_server_private_IP -m comment --comment "redis" -j ACCEPT
[[email protected] ~]# service iptables restart

To protect Redis further, setup authentication which is a built in security feature. This will force clients to authenticate before being granted access. Use a tool such as apg or pwgen to create a secure password. Set the password within Redis by:

[[email protected] ~]# vim /etc/redis.conf
...
requirepass your_secure_password_here
...

[[email protected] ~]# service redis restart

Then test to ensure the password works by:

# This should fail
[[email protected] ~]# redis_cli
127.0.0.1:6379> set key1 10
(error) NOAUTH Authentication required.

# This should work
[[email protected] ~]# redis-cli
127.0.0.1:6379> auth your_secure_password_here
127.0.0.1:6379> set key1 10
OK
127.0.0.1:6379> get key1
"10"

Next, we need to secure the file permissions for Redis. The redis.conf contains the password for redis, so that file shouldn’t be readable by everyone. We also want to lock down the Redis data directory. Lock down the permissions on Redis by:

[[email protected] ~]# chown redis:redis /var/lib/redis
[[email protected] ~]# chmod 700 /var/lib/redis
[[email protected] ~]# chown redis:redis /etc/redis.conf
[[email protected] ~]# chmod 600 /etc/redis.conf
[[email protected] ~]# service redis restart

The Official Redis Administration Guide recommends disabling Transparent Huge Pages (THP). This can be performed live by:

[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
[[email protected] ~]# echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

And disable Transparent Huge Pages (THP) at boot time by entering the following before the ‘exit0’:

[[email protected] ~]# vim /etc/rc.local
...
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
   echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
   echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
...

The Official Redis Administration Guide also recommends setting the following sysctl:

[[email protected] ~]# sysctl vm.overcommit_memory=1
[[email protected] ~]# vim /etc/sysctl.conf
...
vm.overcommit_memory = 1
...

Redis Configurations

Now that Redis is installed and secured, its time to tune it for your application’s needs. There are typically 2 types of Redis cache configurations I see:
Session store
Database cache or full page cache

The session cache allows for a single locations for your application to store sessions that would normally be stored by PHP on the local file system. It is important to ensure that the data is saved to disk so you don’t lose all the sessions between restarts of Redis. This is one of the primary advantages of using Redis over Memcached.

The full page cache is great for caching SQL queries or for serving as a full page cache for applications such as Magento or WordPress, caching images, videos, html, css or js. Generally this type of cache doesn’t need to be persistent across restarts of Redis.

Redis Configurations – Session store

When using Redis as a session store, you want to ensure that the Redis data persists between restarts of Redis. Otherwise your users could be left wondering why their shopping carts all of a sudden vanished. The example below will have disk persistence enabled with a memory limit of 1.5G (1536M).

These settings may or may not work for you! Adjust your settings to meet your environments requirements!

[[email protected] ~]# vim /etc/redis.conf
...
## Ensure disk persistence is enabled
save 900 1
save 300 10
save 60 10000
...
## Set the max memory
maxmemory 1536mb
...

[[email protected] ~]# service redis restart

Redis Configurations – Database cache or full page cache

When using Redis for database caching or as a FPC for applications like WordPress or Magento, I disable disk persistence. This means the cache will only be stored in memory and lost whenever Redis restart. I also set the memory limit to 1.5G (1536M) for starters and adjust accordingly from there. Since I am only storing cached data, I can avoid out of memory issues by allowing Redis to automatically remove the oldest cache entries using the maxmemory-policy allkeys-lru. Read up on the Redis supported eviction policies here.

These settings may or may not work for you! Adjust your settings to meet your environments requirements! Remember, this example assumes everything in Redis can be lost when the service restarts and the eviction policy will remove the least used keys out of all the data. I typically find this works for Magento and WordPress Redis setups:

[[email protected] ~]# vim /etc/redis.conf
...
## Disable disk persistence
#save 900 1
#save 300 10
#save 60 10000
...
## Set the max memory
maxmemory 1536mb
...
## Update the eviction policy
maxmemory-policy allkeys-lru
...

[[email protected] ~]# service redis restart

Multiple Redis Configurations

Redis has the ability to utilize multiple caching configurations with their own settings. The only requirement is that each instance of Redis listens on a unique port, has a unique pid, and of course has its own config and startup script. In the example below, we are going to have 2 Redis instances running called redis-sessions and redis-cache. To avoid confusion, we will disable the original Redis instance.

# Create 2 new configs and modify the values accordingly
[[email protected] ~]# cp /etc/redis.conf /etc/redis-session.conf
[[email protected] ~]# vim /etc/redis-session.conf
...
pidfile /var/run/redis_session.pid
port 6379
logfile /var/log/redis/redis-session.log
dir /var/lib/redis-session
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-session.sock
unixsocketperm 777
...

[[email protected] ~]# cp /etc/redis.conf /etc/redis-cache.conf
...
pidfile /var/run/redis_cache.pid
port 6380
logfile /var/log/redis/redis-cache.log
dir /var/lib/redis-cache
...
# If unixsocket is uncommented, then update to:
unixsocket /var/run/redis/redis-cache.sock
unixsocketperm 777
...

# Create directories and secure the permissions
[[email protected] ~]# mkdir /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chown redis:redis /var/lib/redis-session /var/lib/redis-cache /etc/redis-session.conf /etc/redis-cache.conf
[[email protected] ~]# chmod 700 /var/lib/redis-session /var/lib/redis-cache
[[email protected] ~]# chmod 600 /etc/redis-session.conf /etc/redis-cache.conf

# Create startup files
[[email protected] ~]# cp /etc/init.d/redis /etc/init.d/redis-session
[[email protected] ~]# vim /etc/init.d/redis-session
...
pidfile="/var/run/redis/redis_session.pid"
REDIS_CONFIG="/etc/redis-session.conf"
...
[[email protected] ~]# cp /etc/init.d/redis /etc/init.d/redis-cache
[[email protected] ~]# vim /etc/init.d/redis-cache
...
pidfile="/var/run/redis/redis_cache.pid"
REDIS_CONFIG="/etc/redis-cache.conf"
...

# Stop and disable old instance, start new instances
[[email protected] ~]# service redis stop && chkconfig redis off
[[email protected] ~]# service redis-session start && chkconfig redis-session on
[[email protected] ~]# service redis-cache start && chkconfig redis-session on

# Finally, edit the /etc/redis-session.conf and /etc/redis-cache.conf using the instructions earlier in this article for configuring sessions and db cache.

Client setup

The typical use cases I run into on a day to day basis are clients using Redis for their PHP application. Redis can be used to store cached content, or it can be used to centrally store sessions. Therefore these examples will be PHP focused.

Client setup – General data caching

For storing data, there is nothing that needs to be configured on the client side. The application code itself is what controls storing content within Redis.

Client setup – Storing sessions in Redis

To have Redis act as a central server for sessions, some additional configuration is needed on each client web server. Install php-memcache for your version of PHP. Assuming the default PHP version is installed from the package manager, you can install it by:

[[email protected] ~]# yum install php-pecl-redis
[[email protected] ~]# service httpd graceful
[[email protected] ~]# php -m |grep redis

Then update the php.ini as follows:

session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=your_secure_password_here"

On CentOS and Red Hat servers, depending on what version of PHP was installed and how, you may have to update another file as it will override the php.ini. Only change this if the values exist and are configured for files:

[[email protected] ~]# vim /etc/httpd/conf.d/php.conf
session.save_handler = redis
session.save_path = "tcp://127.0.0.1:6379?auth=your_secure_password_here"

Test to ensure sessions are now being stored in Redis:

[[email protected] ~]# vim /var/www/html/test-sessions.php
<?php
session_start();
?>
Created a session

Then run the following on the command line and confirm the returned numbers increment as shown below:

[[email protected] ~]# curl localhost/test-sessions.php && redis-cli -a your_secure_password_here keys '*' |grep SESSION | wc -l
10
Created a session
11

Troubleshooting

Confirm Redis is online:

[[email protected] ~]# redis-cli ping

How to connect using redis-cli when redis is running on a different server or using a different port:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here

Sometimes you may need to flush the Redis cache. Before doing this, make sure you are connecting to the right instance of Redis since there could be multiple instances running. An example is below:

[[email protected] ~]# redis-cli -h ip_of_redis_server -p port_number_here
FLUSHALL

To get some useful stats on Redis, run:

[[email protected] ~]# redis-cli
info

To get memory specific stats, run:

[[email protected] ~]# redis-cli
info memory
127.0.0.1:6379> info memory
# Memory
used_memory:488315720
used_memory_human:465.69M
used_memory_rss:499490816
used_memory_peak:505227288
used_memory_peak_human:481.82M
used_memory_lua:36864
mem_fragmentation_ratio:1.02
mem_allocator:jemalloc-3.6.0

To increase the memory limit assigned to Redis without restarting the service, the example shown below will increase the memory allocated dynamically from 1G to 2G without a restart of the Redis:

[ro[email protected] ~]# redis-cli
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "1000000000"
127.0.0.1:6379> config set maxmemory 2g
OK
127.0.0.1:6379> config get maxmemory
1) "maxmemory"
2) "2000000000"

Regarding performance issues with Redis, there are a number of factors that need to be account for, too many for this article. Redis published an excellent article that goes into various things that could cause latency with Redis at:
https://redis.io/topics/latency

Magento CE 1.9.x setup on CentOS 6

Setting up a load balanced Magento setup can be a bit daunting. Anyone that has worked with Magento in the past knows that getting the architecture right the first time around is key. The right architecture will vary from solution to solution depending on the needs of the site.

The scalable solution outlined in this document will be build on the Rackspace Cloud, and will have the following server components:

- Domain:
www.example.com

- Load Balancer:  
lb01-http.example.com (With SSL termination)

- Servers
db01.example.com 64.123.123.1 / 192.168.1.1 (Master DB Server)
web01.example.com 64.123.123.3 / 192.168.1.3 (Master Web Server)
web02.example.com 64.123.123.4 / 192.168.1.4 (Slave Web Server)
web03.example.com 64.123.123.5 / 192.168.1.5 (Slave Web Server)

And our setup will utilize the following software to create a scalable and high performing solution:

- Apache 2.2.x with PHP 5.5
- NFS installed on web01 to export the directory /media to the slave web servers
- Lsyncd installed on web01 to sync the documentroot to the slave, and exclude /media, /var, /.git
- Set the 'Admin Base URL' in Magento to point to http://admin.example.com to the master web server, web01
- MySQL 5.6 installed on db01
- Redis 3.x installed on db01 to handle both sessions and provide a centralized cache for all web servers

A special note about web servers: Don’t drive yourself nuts trying to determine which is faster, nginx or Apache. The real performance bottleneck is PHP, and it can be mitigated with a properly configured solution, and using a Full Page Cache (FPC) like Turpentine. I prefer Apache as it is the least complicated one to support in my opinion.

Requirements

Magento CE can be very CPU intensive, even for small to mid size sites. Therefore, you need fast servers with many CPU available. When using the Rackspace Cloud, the minimum server size for lower traffic sites would be 4G General Purpose servers. However as Magento is very CPU intensive, I strongly recommend using 8G General Purpose servers.

The following hard requirements as posted in Magento’s documentation is below for Magento CE 1.9:

Apache 2.x
MySQL 5.6 (Oracle or Percona)
PHP 5.4 or PHP 5.5
Redis or Memcached (For session or cache storage)

The MySQL versions should be noted as Magento does not appear to explicitly state support for MariaDB at this time. They also do not explicitly state support for PHP 5.6. So deviate from these requirements at your own risk!

As per the Magento documentation, if you use MySQL database replication, Magento does not support MySQL statement-based replication. Make sure you use only row-based replication.

[[email protected] ~]# vim /etc/my.cnf
...
binlog-format = ROW
...

Web server prep

Servers involved: All web servers

The prerequisites outlined in here can be found in Magento’s documentation. This guide will assume that you already have Apache and PHP installed on your web servers.

First, apply any needed updates to CentOS 6:

yum update

Now install the required PHP modules for Magento.

# php 5.6 (Unsupported PHP version by Magento!)
yum -y install php56u-xml php56u-mcrypt php56u-gd php56u-soap php56u-devel php56u-mysql php56u-mbstring

# php 5.5
yum -y install php55u-xml php55u-soap php55u-mcrypt php55u-gd php55u-devel php55u-mysql php55u-mbstring

# php 5.4
yum -y install php-mcrypt gd gd-devel php-gd php-mysql mbstring

Increase the PHP memory limit:

vim /etc/php.ini
...
memory_limit = 512M
...

Apache Setup

Servers involved: All web servers

Setting up the Apache vhost for the Magento site is pretty straight forward. Below is a verbose version of the Apache vhost file needed.

First setup the documentroot:

mkdir -p /var/www/vhosts/example.com

Now setup the Apache vhost:

[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf
<VirtualHost *:80>
        ServerName example.com
        ServerAlias www.example.com
        #### This is where you put your files for that domain
        DocumentRoot /var/www/vhosts/example.com

        ### Enable this if you are using a SSL terminated Load Balancer
        SetEnvIf X-Forwarded-Proto https HTTPS=on

	#RewriteEngine On
	#RewriteCond %{HTTP_HOST} ^example.com
	#RewriteRule ^(.*)$ http://www.example.com [R=301,L]

        <Directory /var/www/vhosts/example.com>
                Options -Indexes +FollowSymLinks -MultiViews
                AllowOverride All
		Order deny,allow
		Allow from all
        </Directory>
        CustomLog /var/log/httpd/example.com-access.log combined
        ErrorLog /var/log/httpd/example.com-error.log
        # New Relic PHP override
        <IfModule php5_module>
               php_value newrelic.appname example.com
        </IfModule>
        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn
</VirtualHost>


##
# To install the SSL certificate, please place the certificates in the following files:
# >> SSLCertificateFile    /etc/pki/tls/certs/example.com.crt
# >> SSLCertificateKeyFile    /etc/pki/tls/private/example.com.key
# >> SSLCACertificateFile    /etc/pki/tls/certs/example.com.ca.crt
#
# After these files have been created, and ONLY AFTER, then run this and restart Apache:
#
# To remove these comments and use the virtual host, use the following:
# VI   -  :39,$ s/^#//g
# RedHat Bash -  sed -i '39,$ s/^#//g' /etc/httpd/vhost.d/example.com.conf && service httpd reload
# Debian Bash -  sed -i '39,$ s/^#//g' /etc/apache2/sites-available/example.com && service apache2 reload
##

# <VirtualHost _default_:443>
#        ServerName example.com
#        ServerAlias www.example.com
#        DocumentRoot /var/www/vhosts/example.com
#        <Directory /var/www/vhosts/example.com>
#                Options -Indexes +FollowSymLinks -MultiViews
#                AllowOverride All
#        </Directory>
#
#        CustomLog /var/log/httpd/example.com-ssl-access.log combined
#        ErrorLog /var/log/httpd/example.com-ssl-error.log
#
#        # Possible values include: debug, info, notice, warn, error, crit,
#        # alert, emerg.
#        LogLevel warn
#
#        SSLEngine on
#        SSLCertificateFile    /etc/pki/tls/certs/2016-example.com.crt
#        SSLCertificateKeyFile /etc/pki/tls/private/2016-example.com.key
#        SSLCACertificateFile /etc/pki/tls/certs/2016-example.com.ca.crt
#
#        <IfModule php5_module>
#                php_value newrelic.appname example.com
#        </IfModule>
#        <FilesMatch \"\.(cgi|shtml|phtml|php)$\">
#                SSLOptions +StdEnvVars
#        </FilesMatch>
#
#        BrowserMatch \"MSIE [2-6]\" \
#                nokeepalive ssl-unclean-shutdown \
#                downgrade-1.0 force-response-1.0
#        BrowserMatch \"MSIE [17-9]\" ssl-unclean-shutdown
#</VirtualHost>

Then restart Apache to apply the changes:

[[email protected] ~]# service httpd restart

Magento Installation

Servers involved: web01 only

Download a copy of Magento from their site, and upload it to the /root directory. Once done, move it into place by:

[[email protected] ~]# cd /root
[[email protected] ~]# tar -xvf magento-1*.tar
[[email protected] ~]# cd /root/magento
[[email protected] ~]# cp -a ./* /var/www/vhosts/example.com
[[email protected] ~]# cp -a ./.htaccess /var/www/vhosts/example.com
[[email protected] ~]# chown -R apache:apache /var/www/vhosts/example.com
[[email protected] ~]# crontab -e
*/5 * * * * /bin/bash /var/www/vhosts/example.com/cron.sh

A special note: The cron.sh script only needs to run on the master (admin) web server.

Now browse to your site’s URL, and complete the post installation wizard. When it asks for where you want to store sessions, be sure to specify ‘database’.

Magento admin separation

Servers involved: web01 only

Specifying a master web server for all admin operations is critical for an application like Magento. This allows you to create a subdomain such as ‘http://admin.example.com’, from which all your administrative or backend functions can be run. This helps prevent the age old issue of your images and other work through Magento being uploaded to a slave web server by accident.

Some prefer to do this through Varnish. However in my experience, while Varnish is great for caching, and it is a complete nightmare for admin redirection. So this guide will not be using Varnish. Instead, we’ll use the functionality already provided to us in Magento.

Setting up an admin base URL in Magento CE 1.9 is very simple. First, you need to create an “A” record in DNS to point ‘admin.example.com’ to your master web server. If your using bind, the entry would look like this:

admin.example.com. IN A 64.123.123.3

On web01 only, update Apache’s vhost configuration for the site to include a server alias for the new subdomain, admin.example.com:

[[email protected] ~]# vim /etc/httpd/vhost.d/example.com.conf
<VirtualHost *:80>
...
ServerAlias www.example.com admin.example.com
...
</VirtualHost>

<VirtualHost _default_:443>
...
ServerAlias www.example.com admin.example.com
...
</VirtualHost>

Then restart Apache to apply the change:

[[email protected] ~]# service httpd restart

Finally, log into Magento’s backend control panel, and update the admin base url by:

System -> Configuration -> Admin -> Admin Base URL
Use Custom Admin URL:  Yes
Custom Admin URL:  http://admin.example.com/
Use Custom Admin Path:  No

Lsyncd Setup

Servers involved: web01 only

To ensure that any code changes made on web01 get pushed down to the slave web servers, we are going to install Lsyncd on web01.

On web01 only, install Lsyncd by:

[[email protected] ~]# yum -y install lsyncd
[[email protected] ~]# chkconfig lsyncd on

Now setup the lsyncd configuration by:

[[email protected] ~]# vim /etc/lsyncd.conf
 
settings {
   logfile = "/var/log/lsyncd/lsyncd.log",
   statusFile = "/var/log/lsyncd/lsyncd-status.log",
   statusInterval = 20
}
servers = {
 "192.168.1.4",
 "192.168.1.5"
}
 
for _, server in ipairs(servers) do
sync {
    default.rsyncssh,
    source="/var/www/",
    host=server,
    targetdir="/var/www/",
    excludeFrom="/etc/lsyncd-excludes.txt",
    rsync = {
        compress = true,
        archive = true,
        verbose = true,
        rsh = "/usr/bin/ssh -p 22 -o StrictHostKeyChecking=no"
    }
}
end

Setup the required excludes for Lsyncd:

[[email protected] ~]# vim /etc/lsyncd-excludes.txt
vhosts/example.com/media
vhosts/example.com/var
vhosts/example.com/.git

Finally, start the service

[[email protected] ~]# chkconfig lsyncd on
[[email protected] ~]# service lsyncd start

NFS Setup

Servers involved: NFS server (web01) / NFS Client (all slave web servers)

For load balanced Magento installations, Magento recommends that the /media directory is NFS mounted. On web01, setup NFS by:

[[email protected] ~]# yum install rpcbind nfs-utils -y

Now perform some basic tuning for NFS since the defaults are a bit outdated. Uncomment or add the following variables in /etc/sysconfig/nfs

[[email protected] ~]# vim /etc/sysconfig/nfs
...
RPCNFSDCOUNT=64
RQUOTAD_PORT=875
LOCKD_TCPPORT=32803
LOCKD_UDPPORT=32769
MOUNTD_PORT=892
STATD_PORT=662
STATD_OUTGOING_PORT=2020
...

Open the firewall to allow your private network access to the NFS services. You may have to adjust your rules as my private network resides on eth2. Do not allow this on the public interface without adjusting the source IP’s accordingly!

[[email protected] ~]# vim /etc/sysconfig/iptables
...
-A INPUT -i eth2 -s 192.168.1.0/24 -j ACCEPT
...

[email protected] ~]# service iptables restart

Export the directory to be shared, along with its permissions, in /etc/exports:

[[email protected] ~]# vim /etc/exports
...
/var/www/vhosts/example.com/media 192.168.1.0/24(rw,no_root_squash)
...

Now start the services, and enable them to start at boot time:

[[email protected] ~]# service rpcbind start; chkconfig rpcbind on
[[email protected] ~]# service nfslock start; chkconfig nfslock on
[[email protected] ~]# service nfs start; chkconfig nfs on

Now that the NFS server is ready, the NFS clients now need to be setup to connect. This MUST be performed on each slave server. Install the required packages on the NFS clients by:

[[email protected] ~]# yum install rpcbind nfs-utils -y

Now start the services, and enable them to start at boot time.

[[email protected] ~]# service rpcbind start; chkconfig rpcbind on
[[email protected] ~]# service nfslock start; chkconfig nfslock on
[[email protected] ~]# chkconfig netfs on

Configure the mount point in /etc/fstab:

[[email protected] ~]# vim /etc/fstab

192.168.1.3:/var/www/vhosts/example.com/media  /var/www/vhosts/example.com/media  nfs  vers=3,proto=tcp,hard,intr,rsize=32768,wsize=32768,noatime  0  0

Now create the placeholder directory on the client, mount, and verify it works:

[[email protected] ~]# mkdir /var/www/vhosts/example.com/media
[[email protected] ~]# mount -a
[[email protected] ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup-lv_root
                       14G  1.8G   11G  15% /
tmpfs                 939M     0  939M   0% /dev/shm
/dev/sda1             477M   74M  378M  17% /boot
192.168.1.3:/data      14G  1.9G   11G  15% /var/www/vhosts/example.com/media
[[email protected] ~]#
[[email protected] ~]# grep /data /proc/mounts 
192.168.1.3:/var/www/vhosts/example.com/media /var/www/vhosts/example.com/media nfs rw,noatime,vers=3,rsize=32768,wsize=32768,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.3,mountvers=3,mountport=892,mountproto=tcp,local_lock=none,addr=192.168.1.3 0 0
[[email protected] ~]#
[[email protected] ~]# touch /var/www/vhosts/example.com/media/test-file
[[email protected] ~]# ls -al /var/www/vhosts/example.com/media/test-file 
-rw-r--r-- 1 root root 0 May 5 20:23 /var/www/vhosts/example.com/media/test-file

Be sure to setup the NFS client’s on each slave web server.

Redis Setup

Managing Magento’s cache on a load balancer setup can be a bit of a pain since you would have to log into each server to flush the contents of var/cache whenever you want to empty the cache. This is where a centralized Redis server can come into play. According to Magento’s documentation, they recommend using Redis for session management, and caching. As Magento CE 1.9 supports Redis out of the box, its pretty simple to setup:

On db01, install Redis:

yum install redis30u
chkconfig redis on

Now setup redis to listen on our local network, setup the memory limits, and disable disk caching since we want it to be served out of memory:

vim /etc/redis.conf
...
bind 192.168.1.1
maxmemory 1500mb
maxmemory-policy allkeys-lru
# save 900 1
# save 300 10
# save 60 10000
...

Then start the service:

service redis restart

Now on each web server, install the Redis PHP module:

# PHP 5.4
yum install php54-pecl-redis

# PHP 5.5
yum install php55u-pecl-redis

# PHP 5.6
yum install php56u-pecl-redis

Then restart Apache:

service httpd restart

Finally, update Magento’s configuration to make use of Redis. The redis section is in bold. This only needs to be performed on web01:

cd /var/www/vhosts/example.com/app/etc
cp local.xml local.xml.orig
vim local.xml
...
<config>
    <global>
        <install>
            <date>
        </install>
        <crypt>
            <key>
        </crypt>
        <disable_local_modules>false</disable_local_modules>
        <resources>
            <db>
                <table_prefix><![CDATA[]]></table_prefix>
            </db>
            <default_setup>
                <connection>
                    <host><![CDATA[192.168.1.1]]></host>
                    <username><![CDATA[example]]></username>
                    <password><![CDATA[YOUR_PASSWORD]]></password>
                    <dbname><![CDATA[example]]></dbname>
                    <initStatements><![CDATA[SET NAMES utf8]]></initStatements>
                    <model><![CDATA[mysql4]]></model>
                    <type><![CDATA[pdo_mysql]]></type>
                    <pdoType><![CDATA[]]></pdoType>
                    <active>1</active>
                </connection>
            </default_setup>
        </resources>
        <session_save><![CDATA[files]]></session_save>
  <redis_session>
         <host>192.168.1.1</host>
          <port>6379</port>
           <password></password>
            <timeout>2.5</timeout>
             <persistent></persistent>
              <db>2</db>
               <compression_threshold>2048</compression_threshold>
                <compression_lib>gzip</compression_lib>
                 <log_level>1</log_level>
                  <max_concurrency>6</max_concurrency>
                   <break_after_frontend>5</break_after_frontend>
                    <break_after_adminhtml>30</break_after_adminhtml>
                     <bot_lifetime>7200</bot_lifetime>
                      </redis_session>
               <cache>
               <backend>Mage_Cache_Backend_Redis</backend>
               <backend_options>
               <server>192.168.1.1</server>
               <port>6379</port>
               <persistent></persistent>
               <database>1</database>
               <password></password>
               <force_standalone>0</force_standalone> 
               <connect_retries>3</connect_retries>   
               <read_timeout>10</read_timeout>        
               <automatic_cleaning_factor>0</automatic_cleaning_factor>
               <compress_data>1</compress_data>
               <compress_tags>1</compress_tags> 
               <compress_threshold>20480</compress_threshold> 
               <compression_lib>gzip</compression_lib>
               <use_lua>0</use_lua>
               </backend_options>
               </cache>
    </global>
    <admin>
        <routers>
            <adminhtml>
                <args>
                    <frontName><![CDATA[admin]]></frontName>
                </args>
            </adminhtml>
        </routers>
    </admin>
</config>