Create Encrypted File Container In Linux

Introduction

Most people recommend using Veracrypt to create an encrypted file container. While it is really a good software for this job, I wanted to explore how you can also create an encrypted file container in Linux without installing any third party software. Veracrypt in most cases also provide the same or similar features. However, this solution is native to Linux and as a result is much faster.

You may notice there is alot of things written in this article and that is because it was written over a very long as I learned more about it and is still evolving. Things marked with [Optional] are things that are applicable for small minority of cases that you may ignore and not think twice. You may also not need to worry about [Advanced] sections if you dont have those specific use cases

Pros

• You do not need to install any third party software

• You can use any filesystem you want making it much more flexible

• It has many features you can add such as a recovery key or multiple key slots that allow multiple people to unlock and use the same volume with multiple passwords eliminating the need to share passwords

• It is open source, tried and tested in enterprise environment and secure solution

• You can name it anything and give it any extension

• You do not need to encrypt the whole drive

Cons

• It is not cross platform, although some have been able to use it with WSL2 on Windows recently

• I have not found a way to change the size of the volume after creation. The only way I know is to create a new volume of different size, copy all the content from old volume then delete the old volume

• It takes the full size of the volume even if there is no content on it or it is not yet full. This can be overcome by using a sparse volume but not all filesystems and platforms support it

• LUKS header can take in from 2MB to 64MB depending on whether you are using LUKS1 or LUKS2 but you can place your header in a different place if you want

Notes and Disclaimer

When to run commands as superuser vs when not to

If you see a command prefixed with sudo you are supposed to run those as superuser. You may either choose to use sudo as described or use the actual root user.

Choice of filesystem

In the examples below you will see me using exFAT filesystem. The reason is because I want it to be compatible with EDS Lite application on Android. However, remember that you can use any filesystem of your choice including ext4, btrfs, xfs, whatever you want. Just replace mkfs.exfat with mkfs.ext4 in case of ext4 or mkfs.fat -F32 in case of FAT32 before adding other flags.

If you are having issues with formatting exFAT filesystem below, you may need to install exFAT utilities depending on your Linux distribution. For Ubuntu, you may get them by running the command

sudo apt-get install exfat-fuse exfatprogs

For other distros, search up exfat in the repos and install all the relevant packages. For ext4 or other filesystems you may or may not need to install anything.

Should you use LUKS 1 or LUKS 2

If you have a strict security requirement and have a large volume size to spare, you are absolutely recommended to use LUKS2. However, keep in mind that LUKS2 headers are quiet big, almost as big as 64MB. So when using LUKS2, make sure you have much more storage than 64MB. In order to use luks2, you are supposed to put in --type luks2, however even if you dont put in the --type flag it will by default select LUKS 2.

For most people, LUKS1 is fine. It also has very small header size of around 2MB. It also makes your volume compatible with EDS Lite application. This is what you will see me demonstrating in the examples below. To set luks to luks1 you have to put in --type luks1 otherwise it will select luks2 by default.

KiB/MiB/GiB vs KB/MB/GB

When using dd command, you can either supply file size in KiB or KB. For example, to define a volume size of 4 Gigs you can pass in as 4G which defines 4GiB or 4GB which defines 4GB. File sizes are often defined in KiB or MiB to avoid confusion between whether 1MB = 1024KB or 1MB = 1000KB. It is intuitive for us to think of it in terms of multiples of 1024 so we will stick to using K for Kilobyte and M for megabyte, etc.

Should you use sparse file or not

Sparse files only take up space that it actually needs. This is very useful because the filesize of the container is reduced when it is not full. However, you should keep in mind that not all filesystems support them. It is reported not to work in Apple HFS+ filesystems and the support varies between filesystems. It is however reported to work with NTFS filesystem as well as most Linux native filesystems like ext4, xfs and so on. Learn more in the Arch Wiki

You can create sparse files using either dd or truncate commands. I only demonstrate how to use truncate command to avoid confusion but you will find how to use dd in the linked ArchWiki article. Say for example you wanted to create a volume named myvolume that is of size 3 GiB, ie. the maximum size this volume can hold is 3GiB, you would use the truncate command as so,

truncate -s 3G myvolume

Here you can postfix M to denote size in MB and G to denote size in GB. For example, to create a file size 512MiB, you would specify the size as 512M

Things to rememeber with dd and truncate command

You need to be very careful when running dd command as well as truncate command, be especially sure to double check of= value in case of dd because if the file with that name in that path already exists, dd will just override it without prompting for warning. You may lose your files if you are not careful. Same is true with truncate

Do not mount the same volume multiple times without bind mount

You may accidentally mount the same volume twice in two different mount points. While you are allowed to do so, keep in mind that because they are mounted in two different mountpoints, they are treated as seperate devices by the kernel and do not share the state even though they point to the same physical location on the disk. This causes serious inconsistencies and file corruption in most cases. It is recommended that you only mount a single volume with a single name in a single mount point.

There normally is no reason to do so but if you have a specific reason to have the same volume mounted in multiple places, you can use --bind or --rbind to bind recursively if the volume has further mounted volumes. But keep in mind that you can only use these after you have mounted the volume once somewhere. In this case you choose the mounted location as the source

Example: You want to mount /dev/mapper/myVol into /mnt/myVol and /home/user/myVol

You would first mount /dev/mapper/myVol into /mnt/myVol without bind mount

sudo mount /dev/mapper/myVol /mnt/myVol

You would then bind mount /mnt/myVol into /home/user/myVol

sudo mount --bind /mnt/myVol /home/user/myVol

Get the scripts to do it

The whole process is quiet tedious without using scripts. That is why I have also prepared some scripts you can use to quickly get it working. You can get those scripts from my gitlab

Set it up manually

If you want to do everything manually, perhaps you want to understand every step clearly or want to modify the steps below for your needs than follow along this post

Outline of the process

• Create a volume with dd

• Format that volume with luks

• Open that luks volume

• Format that volume with the filesystem of your choice

• Mount that somewhere

• Use it to store files

• Unmount the volume

• Close the container

• Cleanup in case you made extra directories for mounting

Create a volume with dd

First, create a new volume using dd. You can either use a sparse file as described in Notes and Disclaimer section or a normal file. In case you decide to use a sparse file, you do not need to do any of the below, just create a volume using truncate and move to the next step.

For this example, I am creating a normal non-sparse volume of size 32 MiB named myluks.vol. Then,

• Determine the block size you wish to use

  Some common block sizes are 4K, 8K, 16K, 32K, 64K, 128K, 512K, 1M and 4M. This determines how many blocks are read/written in a single read/write operation. If you choose a very large block size, more data gets written at a time taking more memory but taking less processing power and disk usage. This is good if you have huge files but it wastes memory if you have very small files. There is no wrong answer and it is not necessary to choose this super accurately. This is mostly about performance and memory usage. Most people tend to use either 1M or 4M which is totally fine. For demonstration, I will choose 4K block size because I am going to set it’s volume size to 32M later on where one is not expected to store huge files. This is different from filesystem block size also set later. Also note that block size is written as bs when passed into dd later

• Determine the size of the volume you wish to create

  This is the total size of the volume you wish to create. It can be of any size you want. For this demonstration, I will pick 32 MiB volume. That will be the size I want for my volume.

  It is recommended that you choose a volume size that is a power of 2 such as 2, 4, 8, 16, 32, 64, 128, 256, 512,…

• Convert the volume size to the size unit used by block size

  I chose 32MiB volume size and my block size is 4KiB. My Block size is in KiB and my volume size is in MiB, meaning I will multiply 32 by 1024 which will result in 32768 KiB

  If I had chosen volume size of 32GiB I would instead have multiplied 32 * 1024 * 1024 to convert 32GiB into 33554432KiB

• Divide the resulting volume size by block size

  Now that they use the same storage unit, we can divide them to get count. Since my volume size is 32768 KiB and my block size is 4 KiB, I can divide 32768 KiB by 4KiB to get 8192 which is the resulting count.

  Count is not a whole number, ie. count should not contain decimal point

  You are not allowed to put in a number with a decimal point when supplying either count or bs for dd later down. In case you get a count with a decimal point, you may choose to do one of the following

  • Choose a different block size [Recommended]

    The reason why you are getting a count as a number containing decimal point is because your block size does not equally divide the volume size. So you want to make sure you choose a block size that properly divides the volume size. As an added bonus, you may realize that you can set your block size to 1M or 1K because any number is perfectly divisible by 1

  • Discard the numbers after decimal point

    You may just discard the digits after decimal point. At most you will lose a fraction of the block size. This is negligible loss in storage space and will not matter for most purpose. This, however will matter if you set your block size very big. This is why it is recommended not to set a block size that is very big. This may also cause it to fail to format with the filesystem below due to boundry misalignment so it is not recommended

• Determine the name of the volume

  This can be any name as long as it does not contain spaces. Make sure this file does not already exists or it will be overwritten without any warning resulting in dataloss. For demonstration purpose I will name my volume myluks.vol. You can give any extension or no extension if you prefer. This is passed in as of= into dd

Then you would run this command

dd if=/dev/zero of=myluks.vol bs=4k count=8192 status=progress

Again, make sure that file named myluks.vol does not exist already or it will be overwritten and data in it will be lost

Format that volume with luks

I use LUKS1 as described in Notes and Disclaimer Section

sudo cryptsetup luksFormat --type luks1 myluks.vol

Type in YES in all caps and provide a password. Make sure you provide a good password.

LUKS1 takes up 2M Header size whereas LUKS2 takes up upto 64M. If you wish to use LUKS2, make sure you have enough space for the header and data you wish to store. Ideally, much higher than 64M

[Optional] Use a keyfile

Most of you reading this may be confident with your ability to remember password you set 10 years from now. But if you are like me and you worry about losing your password, you are not alone. In LUKS, as in any other encryption software, losing your password to the vault means losing your data inside that vault for good with no recovery options. Fortunately, luks supports an ability to use a key file to unlock your volume.

Note that keyfile can be any file. It can be a picture, a video, an audio, a text file, anything you want though size may hamper performance when unlocking the volume. But you have to be careful on the following two points

• This key file must never be altered even slightly.

  Absolutely 0 change. For example, if you are using a picture as a key file and one pixel changes, your key file is unusable. Even a single bit flip can cause key file to be unusable

• It must be unreproducable

  This point makes sense if you think of key file like your password. In case of your password, you ask what are the chances of people guessing and recreating the same exact sequence of characters in the same exact position as your password. Same thing here. If you are using something like a picture, a video, an audio, etc then you do not need to worry about this as these files have sufficiently huge amount of bits. This can only be a concern if you are using some text file you wrote yourself as a key file

  Create a file

  As mentioned above, you can use any existing file as your key file. But most people would want to create a seperate key file just for this. This is entirely upto you on how you wish to do it. The key file size you create can be of any size. It is generally recommended that you use atleast a 4 KiB file for decent security. there are two approaches on doing it

  • Also using dd to create a random file as a key file

  You can use dd to create another file that is of any size you want.

  dd if=/dev/random of=keyfile.bin bs=4 count=8
  

  • Using Openssl rand to create a text file with random characters

  As Openssl rand generates text characters rather than just binary bits, it is possible to generate a QR code, print it and put it in a safe. Or, if you really want to go deep, remember that you can write it with a pen and paper and put it safely in a vault. It sounds too far but better safe than sorry if you need to seriously depend on availability of files in the vault long in the future this may be something you may consider

  openssl rand -hex 32 > keyfile.txt
  

  Add that key file to the volume

  You would then add this key file to your volume. Note that in command below, keyfile.bin is the name of the key file and myluks.vol is the preexisting volume where you want to add that key

  sudo cryptsetup luksAddKey myluks.vol keyfile.bin
  

  Ofcourse, you can also remove this key file if you want to from your luks volume. For that you would do

  sudo cryptsetup luksRemoveKey myluks.vol keyfile.bin
  

  Make sure you pass in the correct path to the myluks.vol volume and keyfile.bin

  As is implied, your luks volume is as secure as your key file. If anyone else gets your keyfile, your security is at risk

[Optional] Backup your headers

No LUKS article would be complete without mentioning header backup. And the fact that I have put it under optional may be somewhat controversial considering how important it is that we back up LUKS header. Now, if we were dealing with disk encryption or partition encryption, header backup would be of utmost priority. But since we are dealing with encrypted volume it is not so important as this file would be sitting on top of another filesystem and that means the chance of overriding header is extremely low, though non zero. Regardless, here is how to backup and restore your header

Backup

Backing up your header is as simple as running the command

sudo cryptsetup luksHeaderBackup myluks.vol --header-backup-file headerBackup.bin

Where headerBackup.bin would be the name of the file where header would be set

An important thing to realize is that your password and key file(if you have one) is associated with that header. Meaing that when you restore that header, the password and keyfile used during the time of header creation will replace your current password and key file. For example, you currently created a LUKS container with password password123. Then you backed up your header in a file called headerbackup.bin. Later on, you changed your password of the LUKS container to password456 and you restored headerbackup.bin you backed up earlier, your LUKS container password would be reset to password123 as was when you backed up this header

Restore

To restore headers, you would simply run

sudo cryptsetup luksHeaderRestore myluks.vol --header-backup-file headerBackup.bin

You want to check this command twice. If you specify wrong header backup file, your headers will be overwritten and there are no validations to see if header file is correctly provided

Open that volume

You can use any name to unlock that volume. This name is temporary and will be lost after you close it so you do not have to stress into giving it a good name. Any name that is descriptive enough for the purpose will work. Just do not use spaces in the middle of the name. When opening, you will be asked for the password you set previously. You may also be asked for root password before the luks password

In this case I will name it myVolume

sudo cryptsetup luksOpen myluks.vol myVolume

And if you want to unlock this with a key file instead, you can do so as

sudo cryptsetup luksOpen --key-file keyfile.bin myluks.vol myVolume

In this case, the opened volume is accessible at /dev/mapper/myVolume

If you used another name, ie. myEncVol then it would be accessible at /dev/mapper/myEncVol

What to do if you forgot the name of the volume after opening it

You can use the command

sudo lsblk

Then check the name under loop section

Format the opened volume

Once you opened the LUKS volume, you need to format it with a filesystem to be able to use it. For that, you need to know the name of the opened volume from the previous step. I used the name myVolume so I will use /dev/mapper/myVolume as a device to format. I have also decided to use exFAT as described in Notes and Disclaimer

It is recommended to use the same block size/cluster size that you used in dd as you used in creation of the volume when formatting it for optimal performance. It is not necessary however and things will still work normally even when the block sizes are not aligned. Unfortunately, the flags to provide block size for a filesystem differs from one filesystem to the next so you will have to read the man page to figure that out. In exfat we use the -s option to specify block size in bytes. Since we chose 4K for the block size, when converted to bytes by multiplying with 1024 we get 4096 so we use

sudo mkfs.exfat -s 4096 /dev/mapper/myVolume

Also keep in mind that filesystems also take up header space on top of LUKS. Filesystem header space depends on the filesystem and the cluster size

ERROR: boundary alignment is too big

If you get this error, chances are your block size does not divide the volume size or some other error. This error message is specific to exFAT filesystem but other filesystems display similar error. This may be caused by discarding the decimal digits when setting count value for dd. I would try setting the filesystem cluster size as same as the block size I used in dd and if that didnt work out, I will switch the cluster size to 1M. In some weird case, you may need to reboot the system in order to not have this issue again depending on what version of filesystem driver you are using. But is rarely the case.

Mount the filesystem

• Choose a mount point

  Mount point can be any empty directory in the system at any location owned by any user. Make sure the directory exists, create it if it does not. In this example, I am going to choose /mnt which is already present standard mount point used for this purpose

• Create a folder inside your desired mount point

  It is recommended that you create a folder inside your desired mount point with the name you used to create LUKS volume. This makes it easy to manage multiple volumes that you wish to use. In this example, I will make a folder named myVolume. Note that since /mnt is owned by root user, I need to run this as a superuser. But if it were in a directory your user owned, you should not run this command as a superuser but the user that owned the parent directory

  sudo mkdir /mnt/myVolume
  

• Mount the volume with permission for your user

  If you are using a filesystem that does not support Linux permission and ownership, you would pass in the option -o uid=$(id -u),gid=$(id -g) when mounting. This works well for filesystem such as exfat This, however will cause you an error when mounting a filesystem that supports Linux permission and ownership such as in case of ext4 and other linux native filesystems. In that case, you would mount the directory normally then chown the files within to your user.

  In this case, I am using exFAT filesystem and wish to mount into /mnt/myVolume so I would do it as

  sudo mount -t exfat -o uid=$(id -u),gid=$(id -g) /dev/mapper/myVolume /mnt/myVolume
  

  But say you used ext4 filesystem when formatting the volume. In that case, you would first mount the volume normally as

  sudo mount -t ext4 /dev/mapper/myVolume /mnt/myVolume
  

  followed by

  sudo chown $USER /mnt/myVolume
  

  This means, the actual mount point for my case now is /mnt/myVolume

Use that volume to store sensitive files

The file is mounted in /mnt/myVolume so that means you can open up you file manager, navigate to /mnt/myVolume and start operating on the files there normally. Everything in /mnt/myVolume in this case will be stored encrypted inside that volume you created

Sync disks

If you have recently written alot of data into that volume, it is a good idea to sync the disks after you have completed the copy or move operation to ensure that file buffers were properly written to disks and there will be no file corruption. To do that you can simply run the command

sudo sync

Unmount the volume

After you are done, you can unmount the volume before closing the container

sudo umount /mnt/myVolume

Close the volume

After you are done, you should close the access to the container via its name you used when opening

sudo cryptsetup luksClose myVolume

Remove the folder you created to mount the volume within your chosen mountpoint

I created a folder called myVolume inside /mnt. After I am done working with the volume, I will delete that folder because it is no longer needed. Make sure you are choosing the correct path for the folder you wish to delete. You would adjust this command according to the mount point you used

sudo rm -r /mnt/myVolume

[Optional] Change ownership of /mnt back to root

It is recommended that you do not mount directly into /mnt, especially if you used a linux native filesystem because you may end up unintentionally owning /mnt directory for your user. This is not default behaviour and can cause some issues in some cases. To fix that, you will have to own /mnt as root user

sudo chown root:root /mnt

Normal usage

You are not always going to follow all these steps when using the volume, you can just

• Open the LUKS volume

• Mount the opened volume from /dev/mapper/…

• Start using it

• Unmount the opened volume

• Close the LUKS container

• Remove the folder you made to mount if it was not already present

[Advanced] Using detached header

Normally, LUKS header and encrypted data is stored in the same place. However, you can set it up to store header in a different place than your data. This may be something useful in very small number of situations but I will mention it nontheless as a bonus

You will do this when formatting a volume with LUKS. I could have mentioned this above but since this is optional and very few would be interested in doing, I will instead put it here as a reference

Create a LUKS container with a different header

You can create a LUKS container with a different header as

sudo cryptsetup luksFormat --header myheader.bin myluks.vol

Here, myHeader.bin would be your header file and myluks.vol would be your LUKS volume without header

Opening a LUKS container with a different header

You would then open this LUKS container with a different header as

sudo cryptsetup luksOpen --header myheader.bin myluks.vol myVolume

Everything else from here on is same, you operate and close this container the same way

Conclusion

Even though the steps are quiet cumbersome at first, remember that you can script most of it to make it easier to interact everyday. You can use my scripts from my gitlab linked above or write your own scripts. There are many features in LUKS such as using multiple passwords to open the same volume, storing the header in another location, etc. Everything I have written is just a tip of the iceberg and i suggest you read more on LUKS’s features and so on if you want to learn more