Borgmatic is a free and open source configuration-driven wrapper around Borg, the secure and space-efficient archiver. Borgmatic allows us to orchestrate Borg backups by setting redundancy, rotations, hooks and many other things in a central place: an human-friendly and very well commented configuration file.
In this tutorial we see how to install Borgmatic on some the most used Linux distributions, and how to use it to orchestrate Borg backups.
In this tutorial you will learn:
- How to install Borgmatic
- How to generate a Borgmatic configuration file
- How to orchestrate backups via the Borgmatic configuration file
- How to validate a Borgmatic configuration file
| Category | Requirements, Conventions or Software Version Used |
|---|---|
| System | Distribution agnostic |
| Software | Borgmatic and Borg |
| Other | Privileged access to your Linux system as root or via the sudo command in order to perform system-wide installation of required packages |
| Conventions | # – requires given linux-commands to be executed with root privileges either directly as a root user or by use of sudo command$ – requires given linux-commands to be executed as a regular non-privileged user |
Installation
Being free and open source software, Borgmatic is packaged and available in the official repositories of the most used Linux distributions, and can be easily installed by using their native package managers. To install the “borgmatic” package on Fedora and on Fedora-based distributions, we need to run:
$ sudo dnf install borgmatic
To install the package on Debian and Debian based distributions (Raspberry Pi OS included), instead:
$ sudo apt install borgmatic
On Archlinux and other distributions based on it, we can use the pacman package manager to install the package from the “Extra” repository:
$ sudo pacman -Sy borgmatic
Borgmatic is also distributed as a Python package, therefore can be installed in a “universal” way with “pip”, the Python package manager. This method has the advantage of providing always the latest version of the tool and not requiring administrative privileges if the installation is performed for a specific user or in a virtual environment. To install Borgmatic using pip, we run:
$ pip install --user borgmatic
The command installs the Borgmatic executable under the ~/.local/bin directory, so make sure it is part of your PATH.
Generating the Borgmatic configuration file
Borgmatic is a configuration-driven program: we can fine-tune its behavior just by generating and editing its configuration file. To generate a configuration file from scratch, we use the borgmatic config generate command, and pass the configuration file destination path as argument to the -d option (if the option is omitted, the command tries to generate the file as /etc/borgmatic/config.yaml, therefore it fails if not launched with root privileges). At the user level, Borgmatic searches for a file named config.yaml under the ~/.config/borgmatic directory, therefore, to generate the file in place, we would run:
$ mkdir -p ~/.config/borgmatic && generate-borgmatic-config -d ~/.config/borgmatic/config.yaml
Borgmatic uses a YAML configuration file. Let’s take a look at it.
Defining backup sources and destinations
To specify the files and directories to backup and the backup destinations, which in the Borg world are called repositories, all we have to do is to list them via the source_directories and repositories keys in the configuration file:
location:
# List of source directories and files to backup. Globs and
# tildes are expanded. Do not backslash spaces in path names.
source_directories:
- /home
- /etc
repositories:
- path: ssh://user@backupserver/./sourcehostname.borg
label: backupserver
- path: /mnt/backup
label: local
Under the
source_directories key we define the directories and files we want to include in the backup, while under repositories we enter a list of the local and/or remote Borg repositories to be used as destinations. In recent versions of Borgmatic each repository can be associated to a “label”: this is handy in case we want to reference a specific repository in a convenient way from the command line, as argument to the --repository option. In order to create redundant backups, which is always a good thing, we can provide several destinations: backups are performed sequentially.
Excluding files and directories from the backup
In certain situations, we may want to exclude certain files and directories from a backup. One convenient way to accomplish this task is by using the exclude_patterns key in the Borgmatic configuration file. Here is an example:
exclude_patterns:
- '*.pyc'
- /home/*/.cache
As you can see in the example above, we used simple glob patterns. Any file or directories matched by a pattern is excluded from the backup. As specified in the configuration file, any exclusion pattern where a glob is used must either start with the glob itself or be an absolute path, as in the second example.
Alternatively, exclusion patterns can be read from a file which can be specified by using the exclude_from key:
exclude_from:
- /etc/borgmatic/excludes
Providing the encryption passphrase
As we saw in the tutorial dedicated to Borg, repositories are encrypted using a key, which can itself be stored in the repository (repokey mode) or on the local machine (keyfile mode). When storing the key in a repository, it must be protected by a passphrase. To unlock the key we can:
- put the passphrase directly in the borgmatic configuration file
- specify a command used to obtain the passphrase
To specify the passphrase directly in the Borgmatic configuration file, we can use the encryption_passphrase key. Here is an example:
storage:
encryption_passphrase: "verysecurepassphrase"Writing a password in a plaintext file is not ideal. A slightly more secure alternative is to use the standard output of a command as passphrase. We can do this by using
encryption_passcommand key. Here is an example which involves the use of [pass](https://linuxconfig.org/how-to-organize-your-passwords-using-pass-password-manager), the command-line password manager:
storage:
encryption_passcommand: pass borg-repokey
Rotating backups
The backups rotation strategy to be applied when the “prune” command is executed can be defined via the retention key in the configuration file. We can set how many secondly, minutely, hourly, daily, weekly, monthly and yearly backups should be kept. In the example below we set Borgmatic (and therefore borg, under the hood) to keep 7 daily backups, 4 weekly backups and 6 monthly backups:
retention:
keep_daily: 7
keep_weekly: 4
keep_monthly: 6Borgmatic hooks
One, very handy, Borgmatic functionality is the ability to execute commands and scripts at various points during a backup. Here is a non-exhaustive list of when commands and scripts can be executed:
- Before or after all the actions for all the repositories are performed
- Before or after creating a backup
- Before or after pruning
- Before or after compaction
- Before or after consistency checks
- Before or after extracting a backup
Specific external services can also be used in the “hooks” section. Here is an example. Suppose we want to receive a push notification using the ntfy service when a backup is completed or when it fails. In the configuration file, we use the
ntfy key, and specify the topic we are subscribed to and in what circumstances a notification should be sent; a title, a message and a priority:
hooks:
ntfy:
topic: borgmatic_backups
finish:
title: Backup done!
message: Backup completed successfully!
fail:
title: Backup failed!
message: The backup has failed
priority: max
states:
- finish
- fail
Validating the configuration file
The ones I included above, are just small examples of what can be accomplished by using Borgmatic and its configuration file. When we are done defining our configuration, we want to verify it is valid. In order to do that we can use a dedicated command: validate-borgmatic-config. We pass the path of the configuration file we want to validate as argument to the -c option. In this case, we would run:
$ validate-borgmatic-config -c ~/.config/borgmatic/config.yaml
Creating backups
Once everything is set, to create a backup we can run the following command:
$ borgmatic create --list --stats
The --list and --stats options can be omitted, but are quite useful and correspond to the Borg options with the same name: the former makes so that file details are displayed during the backup, the latter displays statistic on the created archive at the end of the backup:
Archive name: foo-2023-07-02T19:43:56.764318 Archive fingerprint: bd31004d58f51ea06ff735d2e5ac49376901b21d58035f8fb05dbf866566e3c2 Time (start): Sun, 2023-07-02 19:44:11 Time (end): Sun, 2023-07-02 19:44:15 Duration: 4.17 seconds Number of files: 5536 Utilization of max. archive size: 0% ------------------------------------------------------------------------------ Original size Compressed size Deduplicated size This archive: 357.44 MB 340.69 MB 11.22 MB All archives: 2.60 GB 2.28 GB 466.40 MB Unique chunks Total chunks Chunk index: 6729 66061 ------------------------------------------------------------------------------
Pruning backups
To apply the retention policy we set in the Borgmatic configuration file to all repositories, we can use the prune command:
$ borgmatic prune
As you can see, Borgmatic is a wrapper around Borg. It is even possible to specify Borg commands directly by using the
borg sub-command. Just as an example, to run the “list” command directly with Borg, we would run:
$ borgmatic borg list
Conclusions
In this tutorial we learned how to install Borgmatic, a configuration-based wrapper around Borg which makes easier to orchestrate backups and perform various actions at different points in the backup strategy. Instead of having to remember Borg backups option, we can just use the Borgmatic configuration file: this allow us also to replicate our backup strategy with ease.
