Welcome to Hoard!

hoard is a program for backing up files from across a filesystem into a single directory and restoring them later.

Most people will know these programs as "dotfile managers," where dotfiles are configuration files on *nix (read: non-Windows) systems. Files on *nix systems are marked as hidden by starting the file name with a dot (.).

hoard aims to be a little more useful than other dotfile managers:

  1. Many dotfile managers store files in a structure based on their path relative to the user's home directory. This is useful in most cases, but can cause problems when wanted to share files across systems that don't use the same paths, e.g., Windows and Linux. hoard instead namespaces files based on the "Hoard" and "Pile" they are configured in, then relative to the root of the Pile. This makes it easy to backup and restore files to very different locations.

  2. Most dotfile managers do not prevent you from accidentally destructive behavior. See Checks for more information.

Environment

Not to be confused with an environment variable. An Environment is an identifiable system configuration consisting of zero or more each of the following:

  • Operating system
  • Hostname
  • Environment variable
  • Executables in $PATH
  • Existing paths (folders/files) on the system

Multiple Environments can be mixed and matched in Environment Strings when defining what paths to use for a given Pile. Some Environments may be mutually exclusive with certain others.

Pile

A single file or directory with multiple possible places where it can be found, depending on the system configuration. The path to use is determined by the best matching Environment String.

Hoard

A collection of one or more Piles that form a logical unit.

Examples

Consider this configuration snippet (see Configuration File for more explanation):

exclusivity = [
    ["neovim", "vim"],
]

[envs]
[envs.neovim]
    exe_exists = ["nvim", "nvim-qt"]
[envs.unix]
    os = ["linux", "freebsd"]
    env = [
        { var = "HOME" },
        { var = "XDG_CONFIG_HOME" }
    ]
[envs.vim]
    # Detect "vim" if AT LEAST one of `vim` or `gvim` exists in $PATH.
    exe_exists = ["vim", "gvim"]
[envs.windows]
    os = ["windows"]

[hoards]
[hoards.vim]
    [hoards.vim.init]
        "unix|neovim" = "${XDG_CONFIG_HOME}/nvim/init.vim"
        "unix|vim" = "${HOME}/.vimrc"
        "windows|neovim" = "${LOCALAPPDATA}\\nvim\\init.vim"
        "windows|vim" = "${USERPROFILE}\\.vim\\_vimrc"
    [hoards.vim.configdir]
        "windows|neovim" = "${LOCALAPPDATA}\\nvim\\config"
        "windows|vim" = "${USERPROFILE}\\.vim\\config"
        "unix|neovim" = "${XDG_CONFIG_HOME}/nvim/config"
        "unix|vim" = "${HOME}/.vim/config"
  • Environments: neovim, unix, vim, windows; neovim and vim are mutually exclusive.
  • Hoards: just one, called vim, containing two named Piles.
  • Piles: init and configdir; init is the entry config file for a Vim program, while configdir is a directory containing more config files loaded by init.

Take a closer look at the init Pile. There are four possible paths the file can be at, based on a combination of which operating system it is running on and whether Neovim or Vim is installed. The exclusivity line tells Hoard to prefer Neovim if both are present.

File Locations

This page explains what files are generated by hoard and where they can be found.

In general, hoard uses the directories library, using the config_dir and the data_dir of the ProjectDirs struct.

Config Directory

The configuration directory holds the configuration file (config.toml, config.yaml, or config.yml) as well as other local-only configuration data.

  • Linux/BSD: $XDG_CONFIG_HOME/hoard/ or $HOME/.config/hoard/
  • macos: $HOME/Library/Application Support/com.shadow53.hoard/
  • Windows: %AppData%\shadow53\hoard\config\

Config File

The configuration file (config.toml) contains the environment and hoard definitions, along with all related configuration. Follow the link in the previous sentence for more about the configuration file format.

UUID File

The UUID file (uuid) contains a unique identifier for the current system. This is used when performing pre-operation checks. Files relating to this UUID are synchronized between machines using whatever synchronization mechanism you use to synchronize hoards between machines, but nowhere else. This UUID is not used to identify your machines to any service, only the hoard program.

Hoard Data Directory

The hoard data directory contains all backed up hoard files, along with other files that should be synchronized with the hoard files.

  • Linux/BSD: $XDG_DATA_HOME/hoard/ or /home/$USER/.local/share/hoard/
  • macos: $HOME/Library/Application Support/com.shadow53.hoard/
  • Windows: %AppData%\shadow53\hoard\data\

Hoard Files

All files backed up by hoard are stored in the data directory, in a subdirectory called hoards.

The files are organized according to the names of the hoard and pile they are configured under.

As an example, consider the following real configuration:

[hoards.custom_fonts]
    "unix" = "/home/shadow53/.local/share/fonts"

[hoards.fish]
    [hoards.fish.confdir]
        "unix" = "/home/shadow53/.config/fish/conf.d"
    [hoards.fish.functions]
        "unix" = "/home/shadow53/.config/fish/functions"

[hoards.newsboat]
    "unix" = "/home/shadow53/.newsboat/config"

[hoards.qemu]
    [hoards.qemu.script]
        "unix" = "/home/shadow53/.bin/vm"
    [hoards.qemu.configs]
        "unix" = "/home/shadow53/.config/qemu"

These hoards/piles are stored in the following locations:

$data_dir
├─ custom_fonts/
├─ fish
│   ├─ confdir/
│   └─ functions/
├─ newsboat/
└─ qemu/
    ├─ script
    └─ configs/

History Files

There are currently two types of history-related files stored by hoard, both of which are used in pre-operation consistency checks. All history-related files are stored in a subdirectory history/{uuid} in the data directory, where uuid is the generated uuid of the current system.

  • Last Paths: a single file last_paths.json.
  • Operations: date-stamped JSON files with details of which files were modified during a given operation and what the checksum was for each file.

Getting Started

This section will guide you through installing Hoard and setting up your first hoards.

NOTE: The examples use TOML as the config file format. Users looking to use YAML should be able to translate the configuration from TOML. See also this other note.

Installation

This page lists the supported methods of installing hoard.

The recommended method of installation is by downloading a prebuilt executable from the latest release on GitHub. All files are named after the type of system that it can run on. There are many options available, but most people will want one of the following:

  • Windows: hoard-x86_64-pc-windows-msvc.zip
  • Mac:
    • Intel (Older): hoard-x86_64-apple-darwin.tar.gz
    • Apple M1 (Newer): hoard-aarch64-apple-darwin.tar.gz
  • Linux:
    • GNU libc (most distributions, dynamically linked): hoard-x86_64-unknown-linux-gnu.tar.gz
    • MUSL libc (Alpine Linux, statically linked): hoard-x86_64-unknown-linux-musl.tar.gz
  • Modern Android phones: hoard-aarch64-linux-android.tar.gz
    • (you may also want to install Termux)

I've downloaded it, how do I install?

There is no installer for these files -- Hoard is a self-contained executable. Just extract the archive (.zip, .tar.gz), rename the extracted file to hoard, and add it to your $PATH.

What is the $PATH? It is a list of directories that the computer searches for programs when you enter a command on the command line. The process of adding a program to your path is beyond the scope of this guide. Instead, try searching online for "add executable to PATH os name" where os name is the operating system you are running: "Windows", "Mac", "Linux", "Ubuntu", etc.

GNU or MUSL?

Most Linux distributions use GNU libc, so it should be safe to use that version. Because the libc is dynamically linked, security updates to the libc are automatically applied to Hoard and all other programs that use it. This also means that there is a small chance of a libc update breaking programs linked against older versions, though.

MUSL libc is statically compiled into the executable itself, so it can run more or less standalone, without fear of libc breakages. The downside of this is that you do not receive bugfix updates to libc until a newer version of Hoard is released.

You get to decide which one is best to use. For most users (Ubuntu/Debian, etc.), I suggest the GNU libc. For users of fast-moving rolling release systems and systems without GNU libc (Arch, Alpine, etc.), I suggest MUSL.

Cargo

If you have cargo and the Rust toolchain installed, you can install hoard with the following command:

cargo install hoard

Initial Setup

Hoard v0.5.2 added the init subcommand to create the folders and files necessary for Hoard to run.

This command only needs to be run the first time you set Hoard up. After that, including on new machines, it is enough to synchronize the hoard data directory to the new machine and setup or restore the configuration file.

Initializing Hoard

Run hoard init. Everything necessary will be created, including a sample configuration file.

Then, run hoard edit to edit the new configuration file.

Creating the Configuration File

0. Open configuration file

Run hoard edit to open the configuration file in your default editor.

Alternatively, check the File Locations page for the location that the configuration file should be placed. If you are creating the file using your systems File Explorer, you may need to enable hidden files/folders.

1. Choose files to back up

The next step is determining what you are going to back up with Hoard. Common examples are configuration files for various programs and save files for PC games. Just like with Hoard's configuration file, these files are often found in hidden folders, so you may have to do some digging to find them.

For the sake of this guide, we will consider three different programs:

NOTE: The examples use TOML as the config file format. Users looking to use YAML should be able to translate the configuration from TOML. See also this other note.

  1. Hoard itself
  2. Vim and Neovim
  3. Mindustry and Death and Taxes

2. Add configuration for those files

When adding configuration for a specific file or set of files, consider:

  • What to name the hoard and, optionally, the pile or piles within it. See the examples linked above for ideas of how to structure hoards.
  • What conditions must be true for a path to be used. These determine the environments, or envs that you will define.
  • If there are multiple, mutually exclusive conditions that can be true at the same time (see Vim and Neovim for an example). This determines if you need to add anything under exclusivity.
  • Whether the programs use environment variables to determine where to place files, or if it is hardcoded. This will inform whether you use environment variables in the pile path or not.
  • Whether there are files in a directory that you want to ignore when backing up.

2.1: Validate the configuration

When you think you have completed the configuration, double check by running hoard validate. If there are any errors with the configuration file, this command will tell you.

3. Do an initial backup

Once you have validated the configuration, run hoard backup <hoard name>, where <hoard name> is the name of the hoard you just created. Alternatively, you can run hoard backup to back up all configured hoards.

4. Optional: Set up sync

If you want to use Hoard to synchronize files between systems, you'll want to set up some sort of synchronization. Hoard aims to be agnostic to which method is used and only requires that the data files can be found in the expected location. This can be done by synchronizing that directory directly or by creating a symbolic link to another directory.

Possible sync solutions:

  • Syncthing
  • A git repository on any hosting service
  • File synchronization services like Nextcloud/ownCloud, Dropbox, Microsoft Onedrive, etc.

Whatever solution you choose, be aware of the possibility of synchronization conflicts. Hoard has no special logic to prevent synchronization-level conflicts, instead leaving that to the synchronization software itself.

Example: Hoard itself

Let's start with Hoard itself as an example. This allows you to easily share your Hoard configuration across multiple systems with only a little initial setup. Being a single file, it also makes for a simple example.

Depending on the operating system used, the configuration file can be in one of a number of locations. You will want to make entries for each system you plan to use Hoard on. For this guide, we will use Windows and Linux as examples.

1. Choose files to back up

  • Windows: %APPDATA%\shadow53\hoard\config.toml
  • Linux: $XDG_CONFIG_HOME/hoard/ or $HOME/.config/hoard/

The author uses the XDG_CONFIG_HOME path on Linux, but this variable is not always set by default, so this guide will add some logic to cover both cases.

2. Add configuration for those files

Since this example expects Hoard to be used on multiple operating systems, we will create environments for each OS. We will also add an extra environment for when the environment variable XDG_CONFIG_HOME is set.

[envs]
linux = { os = ["linux"] }
windows = { os = ["windows"] }
xdg_config_set = { env = [{ var = "XDG_CONFIG_HOME" }] }

The above configuration uses a shorthand syntax. The following is also valid TOML:

[envs]
[envs.linux]
    os = ["linux"]
[envs.windows]
    os = ["windows"]
[envs.xdg_config_set]
    env = [
        { var = "XDG_CONFIG_HOME" } 
    ]

Now that the environments are defined, we can create the hoard that will contain the configuration file.

[hoards]
[hoards.hoard_config]
    "windows" = "${APPDATA}/shadow53/hoard/config.toml"
    "linux" = "${HOME}/.config/hoard/config.toml"
    "linux|xdg_config_set" = "${XDG_CONFIG_HOME}/hoard/config.toml"

You will notice that the keys "windows", "linux", and "linux|xdg_config_set" are wrapped in double quotes. This is because of the pipe character |, which is not allowed by default in TOML identifiers. The pipe indicates that multiple environments must match -- in this case, linux and xdg_config_set must both match. The quotes around "linux" and "windows" are merely for consistency.

3. Do an initial backup

You can now run hoard backup hoard_config to back up the configuration file, and hoard restore hoard_config to restore the version from the hoard.

Example: Vim and Neovim

This example explores the following concepts:

  • Multiple named piles in a single hoard
  • Mutually exclusive environments
  • Ignoring files by glob pattern

For simplicity, this example assumes only one operating system (Linux) will ever be used. For an example that defines paths based on operating system, see the Hoard Config example.

1. Choose files to back up

While a Vim configuration can live inside a single file, we will consider a situation where there is a directory called config whose contents are included into the main file with the following code:

if has('nvim')
    runtime! config/*.vim
else
    runtime! ~/.vim/config
endif

In this situation, the configuration files are found in the following locations:

  • Vim:
    • Config entrypoint: ${HOME}/.vimrc
    • config directory: ${HOME}/.vim/config
  • Neovim:
    • Config entrypoint: ${XDG_CONFIG_HOME}/nvim/init.vim
    • config directory: ${XDG_CONFIG_HOME}/nvim/config

1.1. Choose files to ignore

For sake of example, let's suppose the config directory contains a number of old, unused files whose names end with .backup. You're going to get around to deleting them eventually, but they might have code you want to keep, just not backed up.

2. Add configuration for those files

As stated above, for simplicity we are assuming that Linux is the only operating system being used -- if it were not, we would need to figure out the paths for other operating systems and include configuration conditional on that. Since we are not worried about that, though, the only environments we care about are whether Vim and/or Neovim are installed:

[envs]
    # Checks for CLI Vim *or* GUI (Gtk+) Vim
    vim = { exe_exists = ["vim", "gvim"] }
    # Checks for CLI Neovim *or* GUI (Qt) Neovim
    neovim = { exe_exists = ["nvim", "nvim-qt"] }

Since it is possible for both Vim and Neovim to be installed on the same system, we need to tell Hoard which one to prioritize. In this case, we will prioritize Neovim:

exclusivity = [
    ["neovim", "vim"]
]

[envs]
    # Checks for CLI Vim *or* GUI (Gtk+) Vim
    vim = { exe_exists = ["vim", "gvim"] }
    # Checks for CLI Neovim *or* GUI (Qt) Neovim
    neovim = { exe_exists = ["nvim", "nvim-qt"] }

Finally, define the actual hoard. We'll call it vim:

[hoards]
[hoards.vim]
[hoards.vim.config]
    # This is the configuration for the vim hoard and is include for
    # demonstration only. For this example, you should use the config
    # *inside* the config_dir pile instead.
    ignore = ["**/*.backup"]
[hoards.vim.init]
    "vim" = "${HOME}/.vimrc"
    "neovim" = "${XDG_CONFIG_DIR}/nvim/init.vim"
[hoards.vim.config_dir]
    # This is configuration just for the vim.config_dir pile
    config = { ignore = ["**/*.backup"] }
    "vim" = "${HOME}/.vim/config"
    "neovim" = "${XDG_CONFIG_DIR}/nvim/config"

NOTE: The name config is reserved for hoard/pile configuration and cannot be used as the name of a hoard or pile. This is why the name config_dir is used above: using config would conflict with the hoard-level configuration block.

We use the glob pattern **/*.backup above to indicate that any file in any subdirectory of config/ with suffix .backup should be ignored. Use *.backup for top-level files only.

3. Do an initial backup

You can now run hoard backup vim to back up your Vim/Neovim configuration, and hoard restore vim to restore the latest backup.

Example: Mindustry and Death and Taxes

This example explores the following concepts:

  • Multiple named piles in a single hoard
  • Mutually exclusive environments
  • How to handle files for flatpak'd applications on Linux

For this example, we will consider two games, installed on both Windows and Linux. Further, we will consider multiple methods of installing each game on each system. This is likely more work than one would do in practice, but this is for sake of example.

1. Choose files to back up

Generally speaking, save files are usually found in one of the following locations:

  • User documents
  • The game's installation directory
  • Some hidden directory (e.g. under $XDG_CONFIG_HOME or $XDG_DATA_HOME on Linux, %APPDATA% on Windows, etc.)

Of the two games we will be using, Mindustry sometimes stores its saves in the installation directory, while Death and Taxes stores its saves in a game-specific subdirectory of a location common to all Unity games.

  • Mindustry:
    • Flatpak: ${HOME}/.var/app/com.github.Anuken.Mindustry/data/Mindustry/saves/saves
    • Linux Itch: ${XDG_DATA_HOME}/Mindustry/saves/saves
    • Linux Steam: ${XDG_DATA_HOME}/Steam/steamapps/common/Mindustry/saves/saves
    • Linux Steam Flatpak: ${HOME}/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Mindustry/saves/saves
    • Windows Itch: ${APPDATA}/Mindustry/saves/saves
    • Windows Steam: ${ProgramFiles(x86)}/Steam/steamapps/common/saves/saves
  • Death and Taxes:
    • Linux: ${XDG_CONFIG_HOME}/unity3d/Placeholder Gameworks/Death and Taxes/Saves
    • Windows: ${USERPROFILE}/AppData/LocalLow/Placeholder Gameworks/Death and Taxes/Saves

2. Add configuration for those files

We'll need environments for the operating systems and for the game stores. For simplicity, we will assume that the XDG variables are always set. We'll also need to specify the order of precedence for the game stores, since all three of them could be installed at once time.

The double square brackets ([[]]) are used to indicate that all of the XDG environment variables are set. If single square brackets ([]) were used, it would mean that at least one must be set.

For more, see the documentation for environments.

exclusivity = [
    ["flatpak_steam", "flatpak_mindustry", "linux_steam", "linux_itch"],
    ["win_steam", "win_itch"],
]

[envs]
[envs.flatpak_mindustry]
    exe_exists = ["flatpak"]
    os = ["linux"]
    path_exists = [
      "/var/lib/flatpak/app/com.github.Anuken.Mindustry",
      "${XDG_DATA_HOME}/flatpak/app/com.github.Anuken.Mindustry",
    ]
[envs.flatpak_steam]
    exe_exists = ["flatpak"]
    os = ["linux"]
    path_exists = [
      "/var/lib/flatpak/app/com.valvesoftware.Steam",
      "${XDG_DATA_HOME}/flatpak/app/com.valvesoftware.Steam",
    ]
[envs.linux_itch]
    os = ["linux"]
    path_exists = ["${HOME}/.itch/itch"]
[envs.linux_steam]
    os = ["linux"]
    exe_exists = ["steam"]
[envs.win_itch]
    os = ["windows"]
    path_exists = ["${LOCALAPPDATA}/itch/itch-setup.exe"]
[envs.win_steam]
    os = ["windows"]
    path_exists = ["${ProgramFiles(x86)}/Steam/steam.exe"]
    
[hoards]
[hoards.game_saves]
[hoards.game_saves.death_and_taxes]
    "linux" = "${XDG_CONFIG_HOME}/unity3d/Placeholder Gameworks/Death and Taxes/Saves"
    "windows" = "${USERPROFILE}/AppData/LocalLow/Placeholder Gameworks/Death and Taxes/Saves"
[hoards.game_saves.mindustry]
    "flatpak_mindustry" = "${HOME}/.var/app/com.github.Anuken.Mindustry/data/Mindustry/saves/saves"
    "flatpak_steam" = "${HOME}/.var/app/com.valvesoftware.Steam/.local/share/Steam/steamapps/common/Mindustry/saves/saves"
    "linux_itch" = "${XDG_DATA_HOME}/Mindustry/saves/saves"
    "linux_steam" = "${XDG_DATA_HOME}/Steam/steamapps/common/Mindustry/saves/saves"
    "win_itch" = "${APPDATA}/Mindustry/saves/saves"
    "win_steam" = "${ProgramFiles(x86)}/Steam/steamapps/common/saves/saves"

3. Do an initial backup

You can now run hoard backup game_saves to back up the game saves, and hoard restore game_saves to restore them.

Command-Line Tool

This section describes the usage and behavior of the command-line tool hoard.

Flags

Flags can be used with any subcommand and must be specified before any subcommand.

  • --help: View the program's help message.
  • -V/--version: Print the installed version of hoard.
  • -c/--config-file: Path to (non-default) configuration file.
  • --data-dir: Path to (non-default) hoards data directory.
  • --config-dir: Path to (non-default) hoards config directory.

Subcommands

hoard backup

hoard [flags...] backup [name] [name] [...]

Back up the specified hoard(s). If no name is specified, all hoards are backed up.

hoard cleanup

hoard [flags...] cleanup

Deletes all extra operation log files that are unnecessary for the related check.

hoard diff

hoard [flags...] diff [-v|--verbose] <name>

Shows a list of all files that differ between the system and the hoard given by <name>. This can detect files that were created, modified, or deleted, locally or remotely.

If -v or --verbose is passed, the output will show unified diffs of text files.

hoard edit

hoard [flags...] edit

Opens the Hoard configuration file in the default editor. This uses $EDITOR when set and the system default handler otherwise.

  • On Linux and BSD, this delegates to xdg-open, which must be installed if $EDITOR is not set.

hoard init

hoard [flags...] init

Ensures the Hoard configuration and data directories exist. If there is no configuration file, one is created.

hoard list

hoard [flags...] list

List all configured hoards by name (sorted).

hoard restore

hoard [flags...] restore [name] [name] [...]

Restore the specified hoard(s). If no name is specified, all hoards are restored.

hoard status

hoard [flags...] status

Displays the current status of every configured hoard:

  • modified locally: all changes are local, and this hoard can be safely backed up with hoard backup.
  • modified remotely: all changes are remote, and this hoard can be safely applied locally with hoard restore.
  • mixed changes: changes are a combination of local and remote, and manual intervention is recommended. Using hoard diff may be useful in reconciling changes.
  • unexpected changes: at least one hoard file appears to have been directly modified instead of using hoard backup. hoard diff may be useful in handling the unexpected change.

hoard upgrade

hoard [flags...] upgrade

Automatically upgrades hoard-related files to newer formats. Old formats may be removed in later versions to help keep the codebase clean.

This currently affects:

hoard validate

hoard [flags...] validate

Attempt to parse the default configuration file (or the one provided via --config-file). Exits with code 0 if the config is valid.

Logging

Output verbosity is controlled by the logging level. You can set the logging level with the HOARD_LOG environment variable. Valid values (in decreasing verbosity) are:

  • trace
  • debug
  • info
  • warn
  • error

The default logging level is info for release builds and debug for debugging builds.

Pre-Operation Checks

To help protect against accidentally overwriting or deleting files, hoard runs some consistency checks prior to running any operations.

To skip running the checks, run hoard with the --force flag. There is not currently a way to disable individual checks.

Last Paths

This check compares the paths used previously with a given hoard to the ones resolved for the current operation. If any of these paths differ, a warning is displayed and the operation(s) canceled.

Remote Operations

By default, hoard logs information about successful operations to a directory that is intended to be synchronized with the main hoards directory. This information is used to determine if a given file was last modified by a remote system. If so, a warning is displayed and the operation(s) canceled.

Configuration File

This section describes the configuration file structure, with examples in TOML.

NOTE: Hoard supports YAML as well as TOML, for those who prefer the former format. This guide assumes that those using YAML already know the format and expects that said users are able to translate from TOML examples to YAML for their own configurations.

All other users should use TOML, as it is the default format and the one used by the author.

Environments

The path used for a given pile depends on the best matching environment(s) for a configured path. This page discusses how to define environments. For how to use them with hoards/piles, see Hoards and Piles.

Environments can be matched on one or more of five possible factors:

  • os: Operating System
  • env: Environment variables
    • Can match on just existence or also a specific value.
  • hostname: The system hostname.
  • exe_exists: Whether an executable file exists in $PATH.
  • path_exists: Whether something exists (one of) the given path(s).

All the above factors can be written using two-dimensional array syntax. That is, ["foo", ["bar, "baz"]] is interpreted as (foo) OR (bar AND baz), in whatever way applies to that given factor.

It is an error to include an AND condition for os or hostname, as a system can only have one of each.

[envs]
[envs.example_env]
    # Matching something *nix-y
    os = ["linux", "freebsd"]
    # Either sed and sh, or bash, must exist
    exe_exists = ["bash", ["sh", "sed"]]
    # Require both $HOME to exist and $HOARD_EXAMPLE_ENV to equal YES.
    # Note the double square brackets that indicate AND instead of OR.
    env = [[
      { var = "HOME" },
      { var = "HOARD_EXAMPLE_ENV", expected = "YES" },
    ]]

Exclusivity

The exclusivity lists indicate names of environments that are considered mutually exclusive to each other -- that is, cannot appear in the same environment condition -- and the order indicates which one(s) have precedence when matching environments.

See the example config file for a more thorough example.

exclusivity = [
    # Assuming all else the same, an environment condition string with "neovim" will take
    # precedence over one with "vim", which takes precedence over one with "emacs".
    ["neovim", "vim", "emacs"]
]

Hoards and Piles

Hoards consist of one or more piles, where each pile is a mapping of environment condition strings to paths on the filesystem.

An environment condition string is one or more environment names separated by pipes. The system must match ALL environments in the string in order for the associated path to be considered.

The following rules determine which path to use for a pile:

  1. The condition string with the most environments wins.
  2. If multiple conditions tie for most environments, the exclusivity list is used to determine if one takes precedence.
  3. If multiple conditions have the same precedence, an error is printed and hoard exits.
  4. If no conditions match, the pile is skipped and a warning is printed.
[hoards]
# This hoard consists of a single anonymous pile
[hoards.simple_hoard]
    # This is "foo" and "bar" separated by a pipe character (`|`).
    # It will use this path if the system matches both environments "foo" and "bar".
    "foo|bar" = "/path/to/a/thing"
    # This path is considered if the system matches the environment "baz".
    # It will use this path if one of "foo" or "bar" doesn't match. Otherwise, "foo|bar"
    # takes precedence because it is a longer condition (more environments to match).
    "baz" = "/some/different/path"

[hoards.complex_hoard]
# This hoard consists of two named piles: "first" and "second".
[hoards.complex_hoard.first]
    "foo|bar" = "/some/path/first"
    "baz" = "/some/different/path/first"
[hoards.complex_hoard.second]
    "foo|bar" = "/some/path/second"
    "baz" = "/some/different/path/second"

Environment Variables

Paths may contain environment variables. Environment variables must be written as ${ENVVAR}, where ENVVAR is the environment variable. As an example, the following hoard could be used to back up a user's Documents folder on Linux and Windows.

[hoards.documents]
    "linux" = "${HOME}/Documents"
    "windows" = "${USERPROFILE}/Documents"

For the user myuser, this expands to the following:

[hoards.documents]
    "linux" = "/home/myuser/Documents"
    "windows" = "C:/Users/myuser/Documents"

If the environment variable does not exist (i.e. is not defined), an error is returned and the operation is canceled.

Limitations

  1. There is no support for default values, i.e. ${MYVAR:-"/some/default"}

Pile Configuration

Pile configuration can be defined at three different levels:

  1. Globally
  2. Per-Hoard
  3. Per-Pile

For a given Pile, any/all three of the levels of configuration are "layered" together, as appropriate for each configuration item:

  • Hashing algorithms use the most-specific layer, or the default if not specified.
  • Ignore patterns are merged and deduplicated.
  • Encryption settings will use the most-specific settings.

Hashing Algorithms

Set hash_algorithm to one of the below strings to manually set which hashing algorithm is used when recording Hoard operations.

  • "sha256" (default): SHA-256 is an older but unbroken algorithm.
  • "md5": MD5 is a quick algorithm but also cryptographically broken. Supported for compatibility with an older operation log format and should be avoided.

Ignore Patterns

Set ignore to a list of glob patterns indicating files and folders to ignore. These lists will be merged across all levels of configuration.

# ... snip env definitions of "foo" and "bar" ...

# Top-level config, applies to all hoards
[config]
    # Ignore the .git folder at any depth
    ignore = ["**/.git"]

[hoards]
[hoards.anon_hoard]
    "foo" = "/some/path"
    "bar" = "/some/other/path"
[hoards.anon_hoard.config]
    ignore = [
        "**/.*", # Ignore all hidden files on Linux/macOS
        "*.log", # Ignore all top-level log files
    ]
[hoards.named_hoard]
[hoards.named_hoard.config]
    ignore = ["ignore-in-named-only"]
[hoards.named_hoard.pile1]
    "foo" = "/some/named/path"
    "bar" = "/another/named/path"

File Permissions

For a general discussion of file/folder permission support in Hoard, including Windows-specific limitations, see this page.

Hoard supports setting permissions separately for files and folders, using file_permissions and folder_permissions, respectively. These can be specified in two different ways: a "mode" and boolean flags.

Mode

A "mode" is an octal (base 8) integer representing read, write, and execute permissions for the owning user, users in the owning "group", and all other users. See the Wikipedia article for more.

You can specify a "mode" in TOML by prefixing the number with 0o. For example, a common file mode is 0o644 (read/write for the owner user, readonly for everyone else).

[config]
    file_permissions = 0o644

If using a "mode" is too confusing, you can also use a set of boolean flags: just set these to true or false:

Flags

Note: only is_writable is supported on Windows. All other flags are ignored. Note 2: "others" in the context of these boolean flags are a combination of the "group" and "other" values from a file "mode".

  • is_readable: the owning user can read the contents of the file or folder. This should not be set to false and is provided for completeness' sake.

  • is_writable: the owning user can modify and delete the file or folder.

  • is_executable: this has different meanings depending on whether it applies to files or folders:

    • true for files means that the user can run the file as an executable program.
    • true for folders means that the user can list the contents of the folder.
    • In short, this should always be true for folders.

  • others_can_read: like is_readable but for non-owner users.

  • others_can_write: like is_writable but for non-owner users.

  • others_can_execute: like is_executable but for non-owner users.

# ... snip env definitions of "foo" and "bar" ...

# Top-level config, applies to all hoards
[config]
    # These represent the current defaults used by Hoard:
    # owner-only access.
    file_permissions = 0o600
    folder_permissions = 0o700

[hoards]
[hoards.anon_hoard]
    "foo" = "/some/path"
    "bar" = "/some/other/path"
[hoards.anon_hoard.config.file_permissions]
    # Equivalent to a 0o644 mode
    is_readable = true
    is_writable = true
    others_can_read = true
[hoards.anon_hoard.config.folder_permissions]
    # Equivalent to a 0o755 mode
    is_readable = true
    is_writable = true
    is_executable = true
    others_can_read = true
    others_can_execute = true

File Permissions in Hoard

Hoard supports the three most popular desktop operating systems: Windows, macOS, and Linux. One of these uses a very different implementation of file permissions compared to the others -- this is why Rust only provides one bit of support for all platforms: whether something is readonly or not.

Previous versions of Hoard ignored this and just hoped things would stay consistent. With the release of 0.5.0, though, Hoard added support for setting file permissions on restore.

Configuration

As of 0.5.0, Hoard supports setting configurable permissions on files and folders on a hoard restore.

When Permissions Are Set

Permissions are set on both backup and restore.

Note: discussion of what permissions are set only apply to Unix-like systems, as Windows only supports readonly, which always defaults to false for the owning user.

Backing Up

When backing up files, all files are given a mode of 0600 and all folders are given a mode of 0700, i.e., owner-only access. This is done to provide a little extra filesystem-based security, since the permissions in the Hoard do not affect the permissions given on restore.

Restoring

When restoring files, all files and folders are given the permissions specified in the most-specific parent pile config. That is, the usual precedence holds, and permissions are not merged.

If no permissions are configured, the defaults are 0600 for files and 0700 for folders.