Encrypting Block Storage In The Cloud

For sensitive information being stored in the cloud outside your direct control, it is critical to encrypt your data at rest. Full disk encryption helps to protect you against unauthorized indivuduals mounting your volume without a key.

It should be noted that LUKS encryption will not protect your data when it is mounted and viewable by your server. A malicious user could in theory break into your server and traverse to that mount point.

My requirements

When writing this guide, I am using the following thought process when implementing LUKS on my Cloud Block Storage volume:

– No keys are to be stored on the server. This is for security purposes since your keys shouldn’t be stored on the server. Would you tape the keys to your Porche on the hood? No. The same logic applies here.
– The volume will not be mounted at boot. This is to prevent the server from stopping the boot process for you to enter in the key. Please note that if the server reboots, you will have to manually log in, mount the volume, and type the passphase for the volume before the system can use it again!

Procedure

This example is going to be specific for CentOS 6.

1. Create your Cloud Block Storage Volumes
In this instance, I am going to be using 2x 100G volumes which are already mounted on my server, and will be setting them up in a RAID 1 configuration.

yum install mdadm cryptsetup-luks cryptsetup-luks-devel
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/xvdb /dev/xvdd

Now confirm the RAID is rebuilding itself by typing:

mdadm --detail /dev/md0

2. Now its time to setup LUKS. This will format your volume, so use caution.

cryptsetup luksFormat /dev/md0

Confirm the contents of the message, then type ‘YES’ in uppercase letters. Then enter a very secure passphrase, and store it somewhere safe. Never store the key on your server!

3. You can verify the results of the encryption process by typing the following:

cryptsetup luksDump /dev/md0

4. Time to mount the encrypted volume and give it a name:

cryptsetup luksOpen /dev/md0 mysecurevolume

5. Finally, lets put a filesystem on it and mount:

mkfs.ext4 /dev/mapper/mysecurevolume
mkdir /opt/mysecurevolume
mount /dev/mapper/mysecurevolume /opt/mysecurevolume

6. You can check the status to ensure its encrypted by typing:

cryptsetup status mysecurevolume

7. Now disable automount so your server won’t hang on boot waiting for the passphrase:

vi /etc/grub.conf
# Append the following at the end of all the kernel lines
rd_NO_LUKS

And your done!

To manually mount the volume after a reboot:

cryptsetup luksOpen /dev/md0 mysecurevolume
mount /dev/mapper/mysecurevolume /opt/mysecurevolume

To manually umount the volume, type:

umount /dev/mapper/mysecurevolume
cryptsetup luksClose mysecurevolume

Final thoughts

Remember, full disk encryption utilizing LUKS is only one part of a defense in depth strategy. No security management system is perfect, but each layer you add will help increase your solutions security footprint.

RHCSA Study Guide – Objective 4 : File Systems

############################
Everything below are my raw notes that I took while attending an unofficial RHCSA training session.  I am posting them here in hopes they will assist others who may be preparing to take this exam.  

My notes are my own interpretation of the lectures, and are certainly not a replacement to classroom training either through your company, or by taking the official RHCSA classes offered through Red Hat.  If you are new to the Red Hat world, I strongly suggest looking into their training courses over at Red Hat.
############################

Filesystem Administration

With partitioning, obviously use fdisk. Granted, apparently something called partprobe is no longer used in RHEL6. Thats great, cause I never used it before. So you will have to reboot to bring the system back up. There is a GUI based tool caused disk utility.

So once the partitioning is done, now create the filesystem:

[root@web01 ~]# mkfs.ext4 /dev/sda2

This can help show you if the disk is dirty:

[root@web01 ~]# tune2fs -l

File system tools:

e2label : view/set filesystem label
tune2fs : view/set filesystem attributes
mount/umount : Mount and un-mount filesystems

EXAM NOTE: Be sure that anything you do on the filesystem, you add it to your /etc/fstab cause the system will be rebooted before it will be graded, so you need to ensure that it works properly upon reboot.

Lab

1.  Using fdisk, create a new 100MB partition
[root@web01 ~]# fdisk /dev/sda
n
e
default
default
n
default
+100M
w

2.  Create a new fs on this partition using ext4, a blocksize of 1k, and a reserve space of 2%.  Confirm settings with tune2fs.  Mount the new fs as /u01, and set it to mount at boot.
[root@web01 ~]# mkfs.ext4 -b 1024 -m 2 /dev/sda5
[root@web01 ~]# mount -t blah and update fstab accordingly

3.  Unmount the /u01 fs and force an integrity check.  Remount the /u01 filesystem.  Use e2label to set the fs label on /u01 to /u01.
[root@web01 ~]# umount /u01
[root@web01 ~]# fsck -f /dev/sda5  # NOTE:  You have to specify the -f to FORCE the fsck.  It will NOT run just because you asked for it.  
[root@web01 ~]# e2label /dev/sda5 /u01
[root@web01 ~]# mount -a
[root@web01 ~]# blkid ; just another way to verify your superblock settings.

EXAM NOTE: This may be on test, but it’ll probably be lvm stuff.

Automount (Autofs)

Autofs monitors a certain directory and can automatically mount a file system when a call is made to files in that directory. It will also unmount the directory in RHEL6 after it hasn’t been touched in 5 minutes.

Its configuration is in:

/etc/auto.master

EXAM NOTE: Will need to know how to tell system which directories to monitor.

/etc/auto.master
path config file
ex.  /misc /etc/auto.misc

This tells automountd to ‘watch’ the /misc pathname for activity, and if activity is observed, consult /etc/auto.misc for instructions.

So for the basic syntax:

path    options   mount device nfs -fstype=nfs,ro  nfsserver:/share/nfs

* This tells automountd to dynamically mount the nfs share to /share/nfs. Autofs will mount stuff as needed.

Lab

1.  Configure your server to automatically mount /share as an NFS share from server1 to /server1/share when a process changes directories there.

[root@web01 ~]# vi /etc/auto.master
...
/server1        /etc/auto.server1
...

[root@web01 ~]# vi /etc/auto.server1
...
share 192.168.1.100:/share
...

[root@web01 ~]# service autofs restart

EXAM NOTE: I would imagine this will be on the test.

Extended Attributes

lsattr - list attributes
chattr - change attributes

EXAM NOTE: Redhat will likely test on the -i flag. So watch out for it.

ACL’s

getfacl
setfacl

You must have the acl mount option set. It’ll work on / since rh does this by default, but you will have to specify this on any new partitions.

[root@web01 ~]# setfacl -m u:bob:w memo.txt  -> Set facls
[root@web01 ~]# setfacl -x g:ru memo.txt -> removes facls
[root@web01 ~]# setfacl -m default:u:bob:w memo.txt -> setfacls

EXAM NOTE: These WILL be on the test.

Quotas

Quotes allow you to limit fs resources to users. Basically disk quotas. To enable, add the following to the mount options:

[root@web01 ~]# vi /etc/fstab
usrquota,grpquota

[root@web01 ~]# quotacheck -mavug
[root@web01 ~]# quotaon -a # Turn on quotas
[root@web01 ~]# edquota -u test # Set limits

EXAM NOTE: These will be on the test.

Lab

1.  Create a quota for the user student with:
- block soft limit of 100M and a hard limit of 150M
- soft inode limit of 30 and a hard inode limit of 100

2.  Create a quota for the group gdm so that its members collectively have:
- a block soft limit of 200M and a hard limit of 300M
- a soft inode limit of 50 and a hard inode limit of 200

Answers:

[root@web01 ~]# vi /etc/fstab # Add the following mount options
usrquota,grpquota

[root@web01 ~]# mount -o remount /
[root@web01 ~]# quotacheck -mavug
[root@web01 ~]# quotaon /home # Turn on quotas
[root@web01 ~]# edquota student # Set limits

# Interesting note:  To do the math quickly on the cli, do:
[root@web01 ~]# echo $((1024*1*100))
[root@web01 ~]# edquota -g gdm

# Set quotas accordingly.
[root@web01 ~]# repquota -g /home

EXAM NOTE: This may be on the exam.

Disk Encryption – LUKS

Quick start for those interested:
http://www.cyberciti.biz/hardware/howto-linux-hard-disk-encryption-with-luks-cryptsetup-command/

[root@web01 ~]# cryptsetup luksFormat   # NOTE:  This will delete all your stuff on disk!!!
[root@web01 ~]# cryptsetup luksOpen  ...  
[root@web01 ~]# cryptsetup luksOpen /dev/sda5 crypt01

This will exist in /dev/mapper/mapname ie. /dev/mapper/crypt01.
# NOTE: The luksformat will prompt for a passphrase, and you can set it to use 8 keys if you like.

Now you will be able to format it:

[root@web01 ~]# mkfs -t ext4 /dev/mapper/crypt01
[root@web01 ~]# mkdir /crypt01
[root@web01 ~]# mount /dev/mapper/crypt01 /crypt01

Now add entry into fstab:

[root@web01 ~]# vi /etc/fstab
...
/dev/mapper/crypt01 /crypt01 ext4 defaults 1 2
...

Once done, now close it (encrypt it) by:

[root@web01 ~]# cryptsetup luksClose /dev/mapper/crypt01

To make this stuff persistent at boot, edit /etc/crypttab as shown below.

1. To make a LUKS encrypted device available at boot time:

[root@web01 ~]# vim /etc/crypttab
mapname device keyfile options

2. To create a keyfile:

[root@web01 ~]# dd if=/dev/urandom of=/etc/keyfile bs=1k count=4
[root@web01 ~]# cryptsetup luksAddKey  /etc/keyfile

3. Add to crypttab

[root@web01 ~]# vi /etc/crypttab
...
/dev/mapper/crypt01 /dev/sda5 [/path/to/keyfile] [option] 
...

EXAM NOTE: Use keyfiles for test. But in practice, use a passphrase, but understand risks involved.

LAB

1.  Create a new 100M physical volume, then set up a luks encrypted ext4 filessystem on the logical volume, which will be persistent across reboots.

2.  Reboot your machine to verify LUKS filesystems prompt for the passphrase and become accessible automatically after bootup

3.  Browse through the man pages on cryptsetup and crypttab

Answers:

1.  Create your 100M logical partition through fdisk
2.  Setup luks stuff
[root@web01 ~]# cryptsetup luksFormat /dev/sda5  # Answer YES, and type your passphrase
[root@web01 ~]# blkid # confirm it setup the type:  cryptoluks
[root@web01 ~]# cryptsetup luksOpen /dev/sda5 crypto  # now enter your password
3.  Now put fs on the device
[root@web01 ~]# mkfs -t ext4 /dev/mapper/crypto
[root@web01 ~]# blkid # You can now see both the raw device, and the crypted device
4.  Setup /etc/fstab
[root@web01 ~]# vi /etc/fstab
...
/dev/mapper/crypto /u02 ext4 default 1 2  # If the test is wonky, set it to 0 0 to prevent fsck.
...
5.  Mount it and your done.
6.  Now create crypttab stuff

# Quick and dirty
[root@web01 ~]# echo -n test > /etc/keyfile  # You need the -n to prevent the newline character
[root@web01 ~]# cryptsetup luksClose /dev/mapper/crypto
[root@web01 ~]# cryptsetup luksOpen /dev/sda5 crypto -d /etc/keyfile # The -d flag forces the key to be used.

# Better way of setting up key - If you don't want to use a pw at all, then do the lukFormat with the -d to specify keyfile.
[root@web01 ~]# dd if=/dev/urandom of=/etc/keyfile bs=1k count=4
[root@web01 ~]# cryptsetup luksAddKey /dev/sda5 /etc/keyfile
# add your original key password
[root@web01 ~]# chmod 400 /etc/keyfile
Now your key works, and so does your passphrase.

[root@web01 ~]# vi /etc/crypttab
crypto /dev/sda5 # If you leave it like this, it'll prompt you for pw at boot
crypto /dev/sda5 /etc/keyfile   # <-- This is how you should do it.

# The method above gives you a secure key, and also a backup passphrase to ensure all is well if you lose your key, you aren't in trouble.

# How to verify all this:
# Confirm your device is unmounted
# This is basically just a way to verify your system will boot most likely.  
[root@web01 ~]# bash
[root@web01 ~]# source /etc/init.d/functions
[root@web01 ~]# init_crypto 1 # This is the function that processes crypttab.  It accepts 0 or 1.  Think of it like mount -a sorta.
[root@web01 ~]# ls -al /dev/mapper
[root@web01 ~]# mount -a
[root@web01 ~]# ls -al /u02

SELinux

Exam note: You can likely leave SElinux disabled or permissive. They will likely not test it at all. It'll be on the RHCE though.

SElinux sits on top of the kernel, telling the kernel what is permitted and what is not. There are 3 levels:

- Disabled : Extentions and hooks are not in kernel
- Permissive : Extension and hooks are there, but if there is a policy violation, the kernel will still allow it.
- Enabled:  Everything there, and blocking accordingly.

Redhat made policies called TARGETED. These are done by groups such as web, mail, ftp, db, etc. Its RHEL's way of making our lives a bit easier. Therefore, by using these targetted polcies, we may just have to fix the files/directories contexts or booleans.

So every process or object has a SELinux context:
- identity:role:domain/type

a.  What identities can use which roles
b.  What roles can enter which domains
c.  What domains can access which types.

Again, RHEL makes this easier and basically just uses the types, nothing else. We can take it further, but that is our choice to make that work.

So in short, SELinux tells the kernel weather or not to allow access to whatever thing.

If you want to view a context for a process, run:

[root@web01 ~]# ps -Z - List the processes contexts
[root@web01 ~]# ls -Z - List the file contexts

To change the context of a file, use:

[root@web01 ~]# chcon -R --reference=/var/www/html file

So what does that mean: Go to this other location (/var/www/html), and apply it to my target (file). So if I put my docroots in /srv, to get SELinux to like this directory, we had to change the context of /srv by:

[root@web01 ~]# chcon -R --reference=/var/www/html /srv

So as long as you know the default location where the contexts reside, you can cheat and just copy the context over to the new location.

All policy violations will be logged to /var/log/audit/audit.log as AVC (access vector cache) denials.
** setroubleshoot is a good tool for reading the output of that log.

Lets say you borked your entire setup, you can reapply the default contexts on all common pathnames. So to restore things, you just do:

[root@web01 ~]# restorecon -R path path...

* NOTE: This will not affect your new stuff like /srv, cause that is not in the default labeling. You can set the semanage stuff (may have to install it), and set the default paths.

restorecon knows about the policies and defaults. chcon only changes things.. that is all chcon knows.

EXAM NOTE: restorecon will not really be needed on the RHCE.. unless you break something hardcore.

There is a graphic tool for selinux: system-config-selinux.
NOTE: You MUST reboot the system when enabling selinux, or disabling it since it mucks with the kernel hooks and stuff.

The config for selinux: /etc/sysconfig/selinux
* This is where you set your enforcing/targetting/disabled, etc. Just the startup mode stuff.

Commands:

getenforce - shows the current SELIinux mode
setenforce - will allow you to change the mode.  ie:  setenforce 0 (dir)
setenforce 0 # Disable selinux temporiatly
setenforce 1 # Enable selinux

NOTE: If the server is completely broken and you cannot even boot, you can disable SELinux in grub by passing the enforcing=0 to the kernel line in grub when booting.

Other troubleshooting tools:

policycoreutils
setroubleshoot

Boolean:
These are basically simple on/off flags for enabling/disabling these:

[root@web01 ~]# getsebool -a |grep httpd  # or whatever.
[root@web01 ~]# setsebool -P blah
# IMPORTANT:  DO NOT FORGET TO SPECIFY THE -P TO MAKE THE CHANGE PERSISTENT ACROSS REBOOTS!

What are some practical uses for selinux:

- Allow you to change the default paths for like where you store db, web, etc, etc.  
- Change the boolean's to allow like public_html directories ie: getsebool -a |grep httpd

Lab

1.  With SELinux enforcing, configure a website to be served from /srv
2.  Dont focus on advanced apache settings, accomplish this in the simplest way possible, change the global documentroot
3.  Populate a simple index.html file.  
4.  The settroubleshoot tool is useful here.  Don't be confused by any typos in the output.

Answer:
Easy enough, just get apache setup, then:

[root@web01 ~]# yum -y install setroubleshootd 
# You will see the stuff needed in /var/log/audit/audit.log or /var/log/messages.
[root@web01 ~]# service auditd restart
[root@web01 ~]# chcon -R --reference=/var/www/html /srv