I am a strong supporter of simplicity and the principle that less is more, but as far as security and performance of information systems is concerned, we must be able to strike a balance between keeping things simple and exposing ourselves as little as possible to potential threats while trying to obtain the maximum performance of all system elements involved.
It is a common practice that the various distributions of Linux, and even the images used to launch Linux virtual server instances as in the case of AWS AMI’s, implement by default an extremely simple partitioning scheme consisting in a single partition that covers the entire disk. It is in this only partition where the root filesystem (/) is mounted and in which all the directories that configure the file hierarchy of a Linux operating system are placed.
But one of the features of Linux is precisely that it allows you to be highly flexible in placing each of these directories on different partitions or on different disks if necessary.
You can go even further and mount them across the network on different machines located in different geographical locations.
And this is not so on a whim, but because each of the directories of the Linux root tree hosts files and libraries intended to perform different tasks with different goals and needs. Thus, having all of them in a single partition can be interesting in the face of simplicity for users who are starting in the Linux world and do not want complications. But if you want to be a serious system administrator and be able to deploy critical systems that are both reliable and secure, you will have no choice but to complicate your life a bit and define a more sophisticated partition structure and mount options scheme.
Different partitions for different tasks
From the entire directory tree hanging from the root directory on a Linux system, there are some directories that deserve special attention as candidates to be mounted on a separate filesystem:
One of the most frequent incidents that occur in a production system, even being easily avoidable, is the 100% filling of a filesystem which results in no more room available to write any new data on it. Similar to what happens in a ship, a compartment that is flooded can be very unpleasant, but if the system is well compartmentalized the problem will not go to major. Conversely, if there is no such separation in watertight compartments, flooding will affect the whole set and the boat will sink. In the same way, if your system only has a single partition and it is fully filled, applications can not continue writing data to disk, so that system behavior turns unpredictable, as it can result in data corruption because cache content can not be written to disk, applications could stop working because they are not able to write on their own temporary files necessary for its operation, or in the worst case the whole system collapses because the operating system can not continue working properly.
The /var directory is the most representative of this situation because it is used to store the system logs and other system database files like the package manager cache. Also, it is typically used by third party applications to store their data files on, like MySQL, Varnish, and many others. Therefore it is normally the one that most data writing receives and in turn the one with the highest growth rate, so if you do not pay attention to it you can easily reach the storage space limit of the partition where /var directory is mounted on. Thus, if it is not separated from the rest it is easy that at some point you can suffer a total system crash.
It may also be desirable to separate other subdirectories inside /var for operational reasons, such as /var/www to have virtual hosts of different web sites hosted on a web server separated from the rest of the system, /var/lib/mysql to isolate your databases from possible /var 100% filling up, /var/log to prevent, given the same scenario, the system from stopping writing logs that are important for security, etc.
Apart from denial of service or data corruption problems mentioned in the previous section, there are inherent security issues that can be addressed simply by setting certain mount options in a filesystem. For example, you can override some common types of attacks by preventing the setuid flag from being set in files permissions using the nosuid mount option, by preventing the creation of device files with nodev option, or by preventing program execution with noexec, which prevents execution permissions (x) from being enabled in any filesystem file.
All of these options are highly recommended especially in the case of /tmp, because by its nature it is a directory that can be accessed without restrictions by any user on the system as it has 777 or -rwxrwxrwx permissions. Therefore it is an easy target for an attacker to install malicious software on it and from there other parts of the system may be compromised.
Regardless of security, it can also be very interesting for system performance to mount the /tmp directory on a filesystem placed in memory instead of on disk. Since many applications use the /tmp directory regularly for normal operation, their performance will be improved if they can read and write files directly from/to RAM, much faster. At the end of this post I present a proposal of partitioning scheme taking into account all these concepts and principles, so you can see below how to mount /tmp in memory.
As in the case of /var, the /home directory can grow up uncontrolled because it is the place where users store their documents, and unless we establish disk usage limits based on quotas, they can easily fill it up completely, either voluntarily or involuntarily. This would cause again the system to collapse if we do not take it to a separate partition. But because it is another place where users can write freely, it also offers potential dangers concerning to security similar to those of /tmp, so it is advisable to set the nosuid and nodev mount options. The activation of noexec option will depend on the use that users will be allowed to make of their personal directory. Of course it would be better a priori not to allow them to run any program, but in many cases it will be legitimate for users to do so, and may be indeed an essential requirement.
What is suitable then is to enable the acl and user_xattr mount options in order to establish more sophisticated access control rules for files and directories and make use of the extended user attributes.
Another interesting feature related to security that division into different partitions will allow us is the possibility to encrypt the contents of /home and thus keep safe our personal documents even if other partitions are not encrypted. It is true that we can find alternative methods to encrypt the contents of personal directory, but having it separated into a different partition facilitates this task and in my opinion is more elegant.
In /boot are hosted both the system boot loader and the different kernels installed. Its contents are not used in normal system operation, only at boot time. It must be able to be read directly by the BIOS so the operating system can boot correctly. This requirement therefore makes it necessary to have a dedicated partition for /boot in some cases, for example if we are using RAID or LVM volumes and our BIOS does not support direct boot from that kind of volumes, or if we need the root filesystem (/) encrypted, since both the bootloader and the kernel can not be encrypted and must be accessible in a simple and direct way by the BIOS. Also if the BIOS is old may have the limitation that the boot partition can only occupy the first 1024 disk cylinders, so in very large disks this limitation will force us to have a small partition at the beginning of the disk on which to mount the /boot directory.
In modern systems that have UEFI replacing the old BIOS it is necessary to have a FAT32 or vfat formatted partition for /boot/efi to comply with the EFI standard and store different boot loaders and drivers in order to boot multiple operating systems.
Thus, although it is not essential in most cases, it is advisable to have a dedicated partition for /boot and another for /boot/efi to have more flexibility when making changes to an existing system. From the perspective of security, it can also be used to apply an additional layer of protection by mounting the /boot and /boot/efi filesystems as read-only with the ro mount, or not mounting them at all (noauto option). As I said above they are only needed at system start up and not during normal operation. This way an attacker could not alter the system booting configuration, and although Linux permissions are already responsible for preventing any modification to users other than root, how I said this is a weak (because if a user gains root access then he can freely mount filesystems and alter mount options) but additional layer of security.
A more effective action would be to install the grub bootloader and the /boot and /boot/efi partitions on a different device that allows physical-read only mode to be enabled, such as an SD card with write-lock switch, or in virtualized environments, immutable volumes whose read-only mode can be controlled from the hypervisor and not from the virtualized instance. This way it would be impossible for an attacker to alter the system boot without having physical access to it or to its hypervisor.
The most interesting reason for separating the /usr filesystem is to be able to set nodev or errors=remount-ro mount options, which will allow it to be mounted in read-only mode if there is an error during the system boot process and thus increase the chances that a faulty system can start. This will give us the opportunity to try to recover it and/or save critical information.
It is important to note that some distributions such as Fedora are making an effort to alter the de facto standard which involved having some directories like /bin or /lib hanging directly from / and taking them through a symbolic link to /usr/lib or /usr/bin. Therefore, in these cases it is not recommended to separate /usr and keep it on the same partition as /.
Imagine you need to store information with very specific characteristics that make advisable to use a file system other than for example the usual ext4. Imagine you need to mount a file server to store large files such as videos or large .tar.gz files. In this case it may be more appropriate to use a filesystem that offers better performance when handling large files, such as XFS. You may also need to use some of the specific features for which other file systems were optimized, such as the ability to take snapshots of your data, transparent data compression or online defragmentation. Given these scenarios you will better want to use other types of filesystems like ZFS or BtrFS.
Although it is possible to place swap areas in files and store them in any directory, the most usual is to allocate a partition specifically for this task, which has the advantage of being able to take the swap area to another more suitable device. For example, if it is very important for us to minimize the performance penalty that occurs when the system runs out of RAM and starts paging or swapping, we could take the swap partition to an SSD disk even though today that means shortening its life. On the other hand, if our computer has enough RAM memory, SSD disks can be assigned to other partitions where their use will be more productive and use a low cost magnetic disk for swap, since its use will be more infrequent as there is no lack of RAM.
From a security perpective, it may be interesting to use some encryption mechanism such as Cryptswap to protect the temporary data hosted in swap area which can contain sensitive information.
Root filesystem /
Finally, and regarding the root filesystem, I would like to point out that not all directories hanging from / are likely to be separated into different partitions. There are directories like /etc, /lib or /bin that should never be placed on separate partitions and all of them should be part of the root filesystem.
Other reasons to design a proper partitioning scheme
Apart from the advantages already explained in previous sections, creating a good partitioning scheme can provide additional possibilities:
It is possible that we are wrong in our initial estimate of the storage space needed for each partition in our partitioning scheme, but at the same time this will provide us with great flexibility when resizing the space if we detect a partition is running out of space or on the contrary it has too much free space that never will be used.
There are filesystems that may need to be mirrored, or even require the use of a distributed file system such as GlusterFS or CephFS.
It is possible to share the contents of a file system in a network by using tools like Samba or NFS. Although it is possible to share only a part of it, ie a subdirectory, this makes it more complicated to administer security and it is recommended to allocate a partition specifically for this purpose.
More powerful quota system
Disk usage quotas are set per user and partition, so to deploy a good quota system it is more interesting to plan a good partitioning scheme to be able for example to assign different quotas to the same user for /home, /tmp, /var, etc.
Apart from the possibility of taking snapshots already commented above, making backups in a system is usually easier if it is divided into different partitions, as there are backup tools that work specifically at the partition level, it is easier know the space required for such backups as it is limited to partition size, certain partitions allow unmounting without affecting the rest of the system in order to prevent further data being written during the operation, which results in consistent backups, etc.
It is recommended to perform a periodical verification of filesystem integrity with tools such as fsck. In the same way as when performing backups, maintenance tasks are facilitated if certain partitions can be unmounted on demand and they can be consistently checked without having to restart the system.
Better use of disk geometry
Disk outer cylinders spin faster than the inner ones, so the former offer better performance than the latter. Thus, it is interesting to place the most used partitions in the outer cylinders. Since the cylinders begin to be numbered from the outside, this means that it is better to put the partitions to be used more intensively at the beginning of the disk and at the end the less used ones. It can also be interesting to place the most used partitions in the middle, so that the heads have to move less and thus increase the life time of disks in the long run.
My partitioning scheme proposal
Finally, I will show you an example of /etc/fstab file and the output of the df -h command corresponding to two different partitioning schemes: one for a medium-sized disk (55 GB) and one for a larger disk size (250 GB). This is for you to use as a starting point to define your own schema and also to illustrate the idea that a standard Linux system does not require a large storage capacity to operate at full performance and with full guarantees of future growth, as the main partitions have a size of a similar order, leaving most of the space of the larger disks available for user’s applications.
Both proposals correspond to real systems based on Ubuntu 16.04.2 LTS operating system.
# /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # <file system> <mount point> <type> <options> <dump> <pass> # / was on /dev/sda2 during installation UUID=7d5ba82c-3085-49e2-b662-6a133ad8e9bf / ext4 defaults,discard,noatime,errors=remount-ro 0 1 # /boot was on /dev/sda3 during installation UUID=7919506c-a13c-4a2c-b27a-8984edd28484 /boot ext4 defaults,noatime 0 2 # /home was on /dev/sda8 during installation UUID=2efc9c4c-5e24-4be5-8076-1135576583f0 /home ext4 defaults,noatime,acl,user_xattr,nodev,nosuid 0 2 # /tmp was on /dev/sda5 during installation #UUID=1e9e3d25-7988-435c-954d-601ea59bd6fd /tmp ext4 defaults,nodev,noexec,nosuid 0 2 tmpfs /tmp tmpfs defaults,nodev,noexec,nosuid,size=512m 0 0 # /usr was on /dev/sda6 during installation UUID=02dd782a-0c87-4f2d-b7d0-b0b1f9bc2982 /usr ext4 defaults,noatime,nodev,errors=remount-ro 0 2 # /var was on /dev/sda7 during installation UUID=b3595283-11ee-4965-8948-b862000678a8 /var ext4 defaults,noatime,nodev,nosuid 0 2 # /var/www was on /dev/sda9 during installation UUID=27eec030-7a45-43a6-8594-d6112e92c345 /var/www ext4 defaults,noatime,acl,user_xattr,nodev,nosuid 0 2 # swap was on /dev/sda4 during installation UUID=47c588d7-4a4a-4f30-8f91-e653cb27e94c none swap sw 0 0 #/dev/mapper/cryptswap1 none swap sw 0 0 UUID=95b03a51-f0d2-4196-bb54-036cba2d166d /var/data ext4 defaults,noatime,acl,user_xattr,nodev,nosuid 0 2
Partitioning proposal for a 55 GB volume
Filesystem Size Used Avail Use% Mounted on udev 2.0G 0 2.0G 0% /dev tmpfs 396M 41M 355M 11% /run /dev/xvda1 7.8G 757M 6.6G 11% / /dev/xvda6 4.8G 1.5G 3.1G 33% /usr tmpfs 2.0G 0 2.0G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 2.0G 0 2.0G 0% /sys/fs/cgroup tmpfs 128M 112K 128M 1% /tmp /dev/xvda7 30G 13G 16G 46% /var /dev/xvda5 3.9G 515M 3.1G 14% /home cgmfs 100K 0 100K 0% /run/cgmanager/fs tmpfs 396M 0 396M 0% /run/user/1000
Partitioning proposal for a volume of 250 GB
Filesystem Size Used Avail Use% Mounted on udev 7,8G 0 7,8G 0% /dev tmpfs 1,6G 97M 1,5G 7% /run /dev/sda2 9,1G 1,8G 6,9G 21% / /dev/sda6 23G 6,8G 15G 32% /usr tmpfs 7,8G 851M 7,0G 11% /dev/shm tmpfs 5,0M 4,0K 5,0M 1% /run/lock tmpfs 7,8G 0 7,8G 0% /sys/fs/cgroup tmpfs 512M 42M 471M 9% /tmp /dev/sda3 2,3G 236M 1,9G 11% /boot /dev/sda7 16G 8,6G 6,6G 57% /var /dev/sda9 124G 103G 15G 88% /var/www /dev/sda8 46G 24G 20G 55% /home /dev/sdb1 917G 828G 44G 96% /var/data tmpfs 1,6G 0 1,6G 0% /run/user/118 tmpfs 1,6G 72K 1,6G 1% /run/user/1000 /home/daniloaz/.Private 46G 24G 20G 55% /home/daniloaz
Macalou - 28/09/2018
Very interesting article.
Bumiga - 15/07/2019
All partitions except /boot should be lvm because of the advantages it brings, snapshot, extend the partition, etc.
Daniel - 15/07/2019
Thanks Bumiga. Most concepts of this article are still valid using LVM or other volume managers. The ultimate goal is to split your filesystem into multiple mount points that allow you to use the most appropriate mount options and ensure that operations on one of them do not affect the others.