Adding an external storage to your server

Introduction

Apart from the monitoring system that ensures that your system's partitions are not too small, YunoHost does not currently deal with the organisation of your partitions and disks.

If you are hosting on an ARM card with an SD card or on a server with a small SSD drive, you may, for reasons of reliability or lack of space, want to add a drive or drives to your server.

Below you will find explanations on how to move your data to a hard disk in a correct way with a minimum of impact on the functioning of YunoHost. This operation can be done during installation or, afterwards, when your storage needs have increased or when you no longer trust your SD card.

Prerequisites

• Have some time at a moment when your server users can accept a shutdown. The steps to be performed, even if they are relatively simple, can sometimes seem technical and require in any case to take your time.

• Know how to connect as root on your system, for example via SSH. (Note: while logged in as admin, you can root with sudo su)

• Know the basic commands cd, ls, mkdir, rm.

• Have a backup in case things don't work out as planned

• Have extra storage (SSD, hard drive, USB stick) connected to your server via USB or SATA

1. Identify directories to be moved

The ncdu / command allows you to browse the folders on your server to see how big they are.

Below is an explanation of some of the paths that can take up weight with some comments to help you reduce their weight or choose to move them.

2. Connect and identify the disk

Start by connecting your disk to your system. You must then identify the name under which the disk is designated by the system.

To do this, use the command :

lsblk

It may return something like :

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda           8:0    0 931.5G  0 disk
└─sda1        8:1    0 931.5G  0 part
mmcblk0     179:0    0  14.9G  0 disk
├─mmcblk0p1 179:1    0  47.7M  0 part /boot
└─mmcblk0p2 179:2    0  14.8G  0 part /

Here, mmcblk0 corresponds to a 16GB SD card (you can see that the mmcblk0p1 and mmcblk0p2 partitions correspond to the /boot partition and the / system partition). The hard drive connected corresponds to sda which is about 1TB, and contains a single sda1 partition which is not mounted (no "MOUNTPOINT").

3. (Optional) Format the disk

This operation is optional if your disk is already formatted with a file system supported by linux (so not NTFS or FAT32).

Let's create a partition on the disk:

fdisk /dev/YOUR_DISK

then enter n, p, 1, Enter, Enter, and w successively to create a new partition.

Check with lsblk that you have your disk containing a single partition.

Before you can use your disk, it must be formatted.

To format the :

mkfs.ext4 /dev/YOUR_DISK1
# then 'y' to validate

Replace YOUR_DISK1 with the name of the first partition on the disk e.g. sda1.

4. Mount the disk

Unlike Windows where disks are accessed with letters (C:/), under Linux, disks are made accessible via the file tree. "Mounting" a disk means making it effectively accessible in the file tree. We will arbitrarily choose to mount the disk in /mnt/hdd but you can name it differently (e.g. /mnt/disk ...).

Let's start by creating the directory :

mkdir /mnt/hdd

Then we can mount the disk manually with :

mount /dev/YOUR_DISK1 /mnt/hdd

(Here, /dev/YOUR_DISK1 corresponds to the first partition on the disk)

5. Mount a /mnt/hdd folder on one of the folders you want to move data from

Here we will consider that you want to move the big data of the applications which are in /home/yunohost.app and the mails on your hard disk.

5.1 Creating subfolders on the disk

To begin with, we create a folder on the hard drive

mkdir -p /mnt/hdd/home/yunohost.app
mkdir -p /mnt/hdd/var/mail

5.2 Switching to maintenance mode

Then, ideally, we switch to maintenance mode the applications that might be writing data.

Example, for nextcloud:

sudo -u nextcloud /var/www/nextcloud/occ maintenance:mode --on

Example, for mail:

systemctl stop postfix
systemctl stop dovecot

5.3 Creating the mount points

Next, we will rename the original folder and create an empty eponymous folder.

mv /home/yunohost.app /home/yunohost.app.bkp
mkdir /home/yunohost.app
mv /var/mail /var/mail.bkp
mkdir /var/mail

We can then use the mount --bind command to mount the folder on our hard drive to the new empty location in the tree.

mount --bind /mnt/hdd/home/yunohost.app /home/yunohost.app
mount --bind /mnt/hdd/var/mail /var/mail

5.4 Copying the data

Next, we copy the data, keeping all the folder and file properties. This operation can take a little time, with another terminal, you can control the evolution by observing the weight associated with the mount point with df -h

cp -a /home/yunohost.app.bkp/. /home/yunohost.app/
cp -a /var/mail.bkp/. /var/mail/

Once this is done, check with ls that the contents are there:

ls -la /home/yunohost.app/
ls -la /var/mail/

5.5 Exiting maintenance mode

From here you can stop maintenance mode, the command below is to be adapted depending on the services you have stopped.

sudo -u nextcloud /var/www/nextcloud/occ maintenance:mode --off
systemctl start postfix
systemctl start dovecot

From this point on, your services are running with their data on disk, so it's time to test to see how much of an impact this has on performance (especially if you are using USB 2.0).

6. Automatically mount on boot

So far we have manually mounted the disk and subfolders. However, it is necessary to configure the system to automatically mount the disk after a boot.

If your tests are successful, you should keep the mount points, otherwise you should hurry up and go back to maintenance first.

To begin with, let's find the UUID (universal identifier) of our disk with :

lsblk -f

Let's add a line to the /etc/fstab file that handles the mounting of disks at boot time. So we open the file with nano :

nano /etc/fstab

Then add these lines to the end of the file:

UUID="cea0b7ae-2fbc-4f01-8884-3cb5884c8bb7" /mnt/hdd ext4 defaults,nofail 0 0
/mnt/hdd/home/yunohost.app /home/yunohost.app none defaults,bind 0 0
/mnt/hdd/var/mail /var/mail none defaults,bind 0 0

(this line must be adapted according to the previous information and choices)

Use Ctrl+X then y to save.

You can then try rebooting the system to check if the disk and subfolders are mounted automatically.

7. Clean up old data

Once your new setup is validated, you can proceed to delete the old data from step 5.3:

rm -Rf /home/yunohost.app.bkp
rm -Rf /var/mail.bkp

Congratulations!

If you have made it this far without damage, you now have a server that takes advantage of one or more storage disks.