diff --git a/LiveCDBackup.org b/LiveCDBackup.org new file mode 100644 index 0000000..b1320ca --- /dev/null +++ b/LiveCDBackup.org @@ -0,0 +1,112 @@ +#+TITLE: Creation of a LiveCD for the restoration of a NixOS system backed up with nix-duplicity-backup + +* Background +** NixOS LiveCD + NixOS provides a module for the creation of a LiveCD through ~config.system.build.isoImage~ [fn:livecd]. + This creates a fully bootable LiveCD with the full build process, starting with GRUB to NixOS proper. + It would probably require a few patches for full disk encryption, alternatively, + it should be possible for the partitition alone to be encrypted instead of FDE. + Furthermore, automating the encryption process might not be fully containable in Nix [fn:nixsecrets] [fn:nixprivate]. + Nix relies on reproducibility yet encryption relies on nonces and initialization -- + a more comprehensive discussion can be found on the Nix Encryption RFC [fn:nixencryption]. + +** nix-duplicity-backup + nix-duplicity-backup is a NixOS module initially authored by [[https://github.com/fgaz][fgaz]] and further modified by [[https://github.com/adrianparvino][adrianparvino]]. + The patches by adrianparvino allows the usage and automation of the creation of + (a) GPG keys, or + (b) password files, + with a common interface. + The current architecture of nix-duplicity-backup requires the + installation of both restoration and backup scripts; + this poses a few problems; + (a) without FDE on the disk, mounting the flash drive will expose the identification files, and + (b) even with FDE on the disk, having the identication files bundled with the restoration script is somewhat sloppy. + nix-duplicity-backup is also currently unsuitable to be used directly for full system backups; + it requires users to manually specify the files to back up -- this is error-prone and tedious. + +* Proposed solution + In addition to the solution proper; this section will also contain solutions to + overcome the shortcomings of the modules specified in the background. + +** Full Disk Encryption on NixOS LiveCD + Full Disk Encryption for the LiveCD is achievable by using LVM instead of squashfs. + Alternatively, we can encrypt individual files in the LiveCD using GPG. + The GPG approach has the advantage of compatibility with ~~, + and the identification files can simply be added with ~isoImage.contents~. + However, the LVM approach has the following advantages: + (1) as the ~.service~ file is located in ~/nix/store~, file encryption is not recommended, but possible, + (2) it requires no special treatment of individual files. + +** nix-duplicity-backup-system + nix-duplicity-backup-system will be a NixOS module which scans for enabled NixOS options. + Take, for example, the postgresql module [fn:postgresql]; + if it is enabled, then backup the file located at ~services.postgresql.dataDir~. + +** Separation of restoration and backup + For this subsection, declare/define is used as in C. + + nix-duplicity-backup configuration should be separated into 3 files -- + (a) ~duplicity-backup-common.nix~ + (b) ~duplicity-backup-backup.nix~ + (c) ~duplicity-backup-restore.nix~ + + The ~duplicity-backup-common.nix~ declares the common interface between backup and restoration; + specifically, the backup directories and the backup destination. + Theoretically, this file should not contain any implementation, sans normalization. + + The ~duplicity-backup-backup.nix~ is one definition of the declared common interface; + it will define the systemd services and timers for periodically backing up the directories to the directories. + + The ~duplicity-backup-restore.nix~ is another definition of the declared common interface; + it will define shell scripts for restoration from backup destinations to their directories. + The restoration script should also allow one to change the target root location, + as the root file system would contain the LiveCD file system rather than the target's file system. + +** Solution proper + We define a new NixOS module which + (a) adds ~nix-duplicity-backup-system~ to the system, and + (b) creates a LiveCD containing ~duplicity-backup-restore~. + The output of ~nix-duplicity-backup-system~ shall be passed as + the input of ~duplicity-backup-restore~. + + The LiveCD is built and encrypted using ~./EncryptedCD~. + Adding everything to the LiveCD's ~/nix/store~, + and only maintaining symlinks to the system, + allows us to remove the need for multiple stages of decryption. + + The LiveCD will be populated with the following files: + (a) The AWS S3 identity files + (b) The partition structure + + Optionally, the LiveCD may also contain the following files: + (a) The duplicity GPG keys + By placing the duplicity GPG keys, + we are able to automate the decryption of the backup. + (b) The duplicity backups + It is also possible to store the backups directly to the USB drive, + allowing it to be restored without internet. + + The partition structure can be generated using heuristics on ~mount~ and ~hardware-configuration.nix~. + + The bootup process will be as follows: +*** Decryption of ~/nix/store~ + Upon bootup, the LiveCD will prompt the user for a decryption key. +*** Rebuilding the partition structure + Using the partition structure provided by ~mount~ and ~hardware-configuration.nix~, + we are able to mimic the file structure of the original system. + Another key is then prompted for the decryption key of the restoration root. +*** [OPTIONAL] Input of the duplicity GPG key + If the GPG identification keys are not saved into ~/nix/store~, + the GPG key is prompted using [fn:interactivesystemd]. +*** Duplicity restore + From here-on, everything should be automatically handled by ~duplicity-backup-restore.nix~. + +[fn:livecd] https://nixos.wiki/wiki/Creating_a_NixOS_live_CD +[fn:nixsecrets] https://github.com/NixOS/nixpkgs/issues/24288 +[fn:nixprivate] https://github.com/NixOS/nix/issues/8 +[fn:nixencryption] https://github.com/edolstra/rfcs/blob/nix-encryption/rfcs/0005-nix-encryption.md + +[fn:postgresql] https://github.com/NixOS/nixpkgs/blob/release-18.09/nixos/modules/services/databases/postgresql.nix + +[fn:interactivesystemd] https://alan-mushi.github.io/2014/10/26/execute-an-interactive-script-at-boot-with-systemd.html +[fn:nixosencryptedroot] https://gist.github.com/martijnvermaat/76f2e24d0239470dd71050358b4d5134 diff --git a/USING.org b/USING.org new file mode 100644 index 0000000..b68bb55 --- /dev/null +++ b/USING.org @@ -0,0 +1,124 @@ +#+TITLE: nix-duplicity-backup +* Prerequisites and installation + + The canonical installation of nix-duplicity-backup is S3+Password. + An alternative to S3 is rsync/SSH, however this is currently disabled. + An alternative to Password login is the usage of GPG keys. + Any S3 bucket will work provided that ACL permissions are granted to your user. + +* Canonical Configuration + + As an example, nix-duplicity-backup, the repository will be backed up. + An S3+Password configuration is used by default. + First, create a ~duplicity-backup-config.nix~ beside ~configuration.nix~: +#+BEGIN_src nix + # This serves as a configuration file, no backups will be created, + # and this is a noop, sans assertions checking and duplicity key generation. + { + # This loads the services.duplicity-backup options. + imports = [ ]; + + services.duplicity-backup = { + # This enabled interpretation of the duplicity-backup config, + # specifically assertions checking and duplicity-gen-keys. + enable = true; + + # Use passphrase instead of GPG keys + usePassphrase = true; + + # Add an archive to duplicity-backup + archives.nix-duplicity-backup = { + # The S3(or SSH) instance to upload the backups to + destination = s3://s3.REGION.amazonaws.com/BUCKETNAME/nix-duplicity-backup; + + # A directory or file to back up + directory = ; + }; + }; + } +#+END_src + +* Credentials management + + Under default configurations, + ~${envDir}~ is located in ~/var/keys/duplicity/env~, + ~${pgpDir}~ is located in ~/var/keys/duplicity/gnupg~. + + With an S3 backend, ~duplicity-gen-keys~ will ask for + ~AWS_ACCESS_KEY_ID~ and ~AWS_SECRET_ACCESS_KEY~. + If you have previously used the AWS CLI, + then these can be found under ~$HOME/.aws/credentials~. + Otherwise, you need to check under + IAM > Users > [your user] > Security Credentials + on the Amazon AWS console. + Access key secrets are only shown upon creation, + so if you don't have an existing secret, + you'll have to generate a new access key ID. + These are statefully stored under ~${envDir}/10-aws.sh~ + + With passphrase enabled, it will prompt for a passphrase, + and store it under ~${envDir}/20-passphrase.sh~. + + These are stored as Bash files, allowing you to load it imperatively using: +#+BEGIN_src bash + for i in ${envDir}/*; do + . $i + done +#+END_src + + With GPG enabled, ~duplicity-gen-keys~ will generate GPG keys + and store it under in ~${pgpDir}~. + These can be loaded into your environment using +#+BEGIN_src bash + export PGP_HOME_DIR=${pgpDir} +#+END_src bash + +* Enabling backups + + Backups can finally be enabled by adding the following to your ~configuration.nix~: +#+BEGIN_src nix + { + imports = [ ./duplicity-backup-config.nix ]; + + # Adds the backup services and timers to systemd for periodic backups. + services.duplicity-backup.enableBackup = true; + } +#+END_src + + To verify that everything works, run ~systemctl start duplicity-nix-duplicity-backup~. + +* Configuration + + More granular configurations are possible: +#+BEGIN_src nix + { + services.duplicity-backup = { + archives.nix-duplicity-backup = { + # Use GPG keys instead of passphrase login + usePassphrase = false; + + archives.nix-duplicity-backup = { + # Defaults to "01:15" + # This makes the backups run hourly instead of 01:15 localtime. + # More info in `man 7 systemd.time`, section CALENDAR EVENTS. + period = "hourly"; + + # Exclude files containing the name "secret" from being uploaded. + excludes = [ "*secret*" ]; + + # However, allow files containing the name "code_to_handle_secret" to be uploaded. + includes = [ "*code_to_handle_secret*" ]; + + # Only use a maximum bandwidth of 1 MB/s. + maxbw = 1 * 1000 * 1000; + + # Use a full backup every week instead of every month. + fullIfOlderThan = "7D"; + + # And only keep 2 weeks worth of full backups + removeAllButNFull = 2; + }; + }; + }; + } +#+END_src diff --git a/duplicity-backup-backup.nix b/duplicity-backup-backup.nix new file mode 100644 index 0000000..f9842ec --- /dev/null +++ b/duplicity-backup-backup.nix @@ -0,0 +1,82 @@ +{ pkgs, config, lib, options, ... }: + +with lib; + +let + gcfg = config.services.duplicity-backup; +in +{ + imports = [ ./duplicity-backup-common.nix ]; + + options = { + services.duplicity-backup.enableBackup = mkEnableOption "periodic duplicity backups backup tools"; + services.duplicity-backup.archives = mkOption { + type = types.attrsOf (types.submodule ({ name, config, ... }: { + options = { + backupService = mkOption { + type = types.attrs; + }; + }; + + config.backupService = { + description = "Duplicity archive ${name}"; + + requires = [ "network-online.target" ]; + after = [ "network-online.target" ]; + + serviceConfig = { + Type = "oneshot"; + IOSchedulingClass = "idle"; + NoNewPrivileges = "true"; + CapabilityBoundingSet = [ "CAP_DAC_READ_SEARCH" ]; + PermissionsStartOnly = "true"; + }; + + script = '' + for i in ${gcfg.envDir}/*; do + source $i + done + + mkdir -p ${gcfg.cacheDir} + chmod 0700 ${gcfg.cacheDir} + + ${pkgs.duplicity}/bin/duplicity \ + --archive-dir ${gcfg.cacheDir} \ + --name ${name} \ + --gpg-options "--homedir=${gcfg.pgpDir}" \ + --full-if-older-than ${config.fullIfOlderThan} \ + '' + optionalString (config.allowSourceMismatch) ''--allow-source-mismatch \ + '' + optionalString (!gcfg.usePassphrase) ''--encrypt-key "Duplicity Backup" \ + '' + '' + ${concatStringsSep " " (map (v: "--include '${v}'") config.includes)} \ + ${concatStringsSep " " (map (v: "--exclude '${v}'") config.excludes)} \ + ${config.directory} \ + ${config.destination} + '' + optionalString (config.removeAllButNFull != null) '' + ${pkgs.duplicity}/bin/duplicity remove-all-but-n-full ${toString config.removeAllButNFull} \ + --archive-dir ${gcfg.cacheDir} \ + --name ${name} \ + --gpg-options "--homedir=${gcfg.pgpDir}" \ + --full-if-older-than ${config.fullIfOlderThan} \ + --force \ + ${config.destination} + ''; + }; + })); + }; + }; + + config = mkIf (gcfg.enable && gcfg.enableBackup) { + systemd.services = mapAttrs' (name: archive: + nameValuePair "duplicity-${name}" archive.backupService + ) gcfg.archives; + + # Note: the timer must be Persistent=true, so that systemd will start it even + # if e.g. your laptop was asleep while the latest interval occurred. + systemd.timers = mapAttrs' (name: cfg: nameValuePair "duplicity-${name}" + { timerConfig.OnCalendar = cfg.period; + timerConfig.Persistent = "true"; + wantedBy = [ "timers.target" ]; + }) gcfg.archives; + }; +} diff --git a/duplicity-backup-common.nix b/duplicity-backup-common.nix new file mode 100644 index 0000000..171fda4 --- /dev/null +++ b/duplicity-backup-common.nix @@ -0,0 +1,351 @@ +{ pkgs, config, lib, ... }: +with lib; +let + gcfg = config.services.duplicity-backup; + + duplicityGenKeys = pkgs.writeScriptBin "duplicity-gen-keys" ('' + #!${pkgs.stdenv.shell} + + usage() + { + cat <&2 && exit 4 + PROFILE= + [ "$1" == "--aws" ] && shift && PROFILE="$1" + AWS_PROFILE="$PROFILE" + esac + shift + done + + writeVar() { + VAR="$1" + FILE="$2" + + touch "$FILE" + printf 'export %s=%q\n' "$VAR" "''${!VAR}" >> "$FILE" + } + + prompt() { + VAR="$1" + + [ "$2" = "SECRET" ] && stty -echo + printf '%s=' "$VAR" 1>&2 + IFS= read -r "$VAR" + [ "$2" = "SECRET" ] && stty echo + [ "$2" = "SECRET" ] && printf '\n' 1>&2 + } + + if [ -z "''${UPDATE+SET}" ]; then + if [ -e ${escapeShellArg gcfg.envDir} ]; then + printf "The environment directory(%s) exists. Use --update to archive it." ${escapeShellArg gcfg.envDir} 1>&2 + exit 1 + elif [ -e ${escapeShellArg gcfg.pgpDir} ]; then + printf "The PGP home directory(%s) exists. Use --update to archive it." ${escapeShellArg gcfg.pgpDir} 1>&2 + exit 1 + fi + fi + + NEW_ENV=${escapeShellArg (gcfg.envDir + ".d")} + NEW_ENV="$NEW_ENV/$(date -Iseconds)" + NEW_PGP=${escapeShellArg (gcfg.pgpDir + ".d")} + NEW_PGP="$NEW_PGP/$(date -Iseconds)" + + umask u=rwx,g=,o= + mkdir -p "$NEW_ENV" + mkdir -p "$NEW_PGP" + umask 0022 + + cleanup () { + rm -fr "$NEW_ENV" + rm -fr "$NEW_PGP" + } + trap cleanup EXIT + + AWS_FILE=$(eval echo "~$SUDO_USER/.aws/credentials") + if [ -e "$AWS_FILE" -a -z "''${AWS_PROFILE+SET}" ]; then + printf 'AWS credentials file(%s) exists. Use [ --no-aws | --aws profile ].\n' "$AWS_FILE" 1>&2 + printf 'Available profiles:\n' 1>&2 + sed -n '/^\[\(.*\)\]$/s//\1/p' 1>&2 < "$AWS_FILE" + exit 2 + fi + + if [ -e "$AWS_FILE" -a -n "$AWS_PROFILE" ]; then + { read -r AWS_ACCESS_KEY_ID; + read -r AWS_SECRET_ACCESS_KEY; + } < <(sed -n '/^\['"$AWS_PROFILE"'\]/,/^\[.\+\]/{ # Get section that starts with $AWS_PROFILE + /^aws_access_key_id *= */ s//0 /p; # Extract AWS_ACCESS_KEY_ID + /^aws_secret_access_key *= */s//1 /p; # Extract AWS_SECRET_ACCESS_KEY + }' < "$AWS_FILE" | sort | cut -d' ' -f2-) + else + prompt AWS_ACCESS_KEY_ID + prompt AWS_SECRET_ACCESS_KEY SECRET + fi + + writeVar AWS_ACCESS_KEY_ID "$NEW_ENV/10-aws.sh" + writeVar AWS_SECRET_ACCESS_KEY "$NEW_ENV/10-aws.sh" + '' + (if gcfg.usePassphrase + then '' + prompt PASSPHRASE SECRET + writeVar PASSPHRASE "$NEW_ENV/20-passphrase.sh" + '' + else '' + ${pkgs.expect}/bin/expect << EOF + set timeout 10 + + spawn ${pkgs.gnupg}/bin/gpg --homedir "$NEW_PGP" --generate-key --passphrase "" --pinentry-mode loopback + + expect "Real name: " { send "Duplicity Backup\r" } + expect "Email address: " { send "\r" } + expect "Change (N)ame, (E)mail, or (O)kay/(Q)uit? " { send "O\r" } + + expect "pub" # Required to flush the last command + + interact + EOF + '') + '' + trap EXIT + + rm ${escapeShellArg gcfg.envDir} + rm ${escapeShellArg gcfg.pgpDir} + + ln -s "$NEW_ENV" ${escapeShellArg gcfg.envDir} + ln -s "$NEW_PGP" ${escapeShellArg gcfg.pgpDir} + ''); + + mkSecurePathsOption = + { description + , default + }: + mkOption { + inherit description default; + + type = with types; listOf (either path string); + apply = xs: map (x: if builtins.typeOf x == "path" then toString x else x) xs; + }; + + mkSecurePathOption = + { description + , ... + }@args: + mkOption ({ + inherit description; + + type = with types; either path string; + apply = x: if builtins.typeOf x == "path" then toString x else x; + } // lib.optionalAttrs (args ? default) { + inherit (args) default; + }); +in +{ + options = { + services.duplicity-backup = { + enable = mkEnableOption "periodic duplicity backups"; + + usePassphrase = mkOption { + type = types.bool; + default = false; + description = '' + Use passphrase instead of keys + ''; + }; + + rootDir = mkSecurePathOption { + default = /var/keys/duplicity; + description = '' + Directory of bash scripts to `source`, + currently used for declaring AWS keys and secrets + ''; + }; + + envDir = mkSecurePathOption { + default = gcfg.rootDir + "/env"; + description = '' + Directory of bash scripts to `source`, + currently used for declaring AWS keys and secrets + ''; + }; + + pgpDir = mkSecurePathOption { + default = gcfg.rootDir + "/gnupg"; + description = '' + Directory of bash scripts to `source`, + currently used for declaring AWS keys and secrets + ''; + }; + + cacheDir = mkSecurePathOption { + default = /var/cache/duplicity; + description = '' + The cache allows duplicity to identify previously stored data + blocks, reducing archival time and bandwidth usage. + ''; + }; + + archives = mkOption { + type = types.attrsOf (types.submodule ({ ... }: + { + options = { + destination = mkOption { + type = types.string; + default = ""; + example = "rsync://user@example.com:/home/user"; + description = '' + ''; + }; + + removeAllButNFull = mkOption { + type = types.nullOr types.int; + default = null; + example = 2; + description = '' + Only keep the given amount of full backups, + useful for pruning old full backups + which are too outdated to be useful. + ''; + }; + + fullIfOlderThan = mkOption { + type = types.str; + default = "1M"; + example = "1D"; + description = '' + Use full backup when fullIfOlderThan time has passed. + + The format is described in + duplicity + 1. + ''; + }; + + period = mkOption { + type = types.str; + default = "01:15"; + example = "hourly"; + description = '' + Create archive at this interval. + + The format is described in + systemd.time + 7. + ''; + }; + + directory = mkSecurePathOption { + description = "File system path to archive."; + }; + + allowSourceMismatch = mkOption { + type = types.bool; + default = false; + }; + + excludes = mkSecurePathsOption { + default = []; + description = '' + Exclude files and directories matching these patterns. + ''; + }; + + includes = mkSecurePathsOption { + default = []; + description = '' + Include only files and directories matching these + patterns (the empty list includes everything). + + Exclusions have precedence over inclusions. + ''; + }; + + maxbw = mkOption { + type = types.nullOr types.int; + default = null; + description = '' + Abort archival if upstream bandwidth usage in bytes + exceeds this threshold. + ''; + }; + + maxbwRateUp = mkOption { + type = types.nullOr types.int; + default = null; + example = literalExample "25 * 1000"; + description = '' + Upload bandwidth rate limit in bytes. + ''; + }; + + maxbwRateDown = mkOption { + type = types.nullOr types.int; + default = null; + example = literalExample "50 * 1000"; + description = '' + Download bandwidth rate limit in bytes. + ''; + }; + }; + } + )); + + default = {}; + + example = literalExample '' + { + nixos = + { directory = /home; + }; + + gamedata = + { directory = /var/lib/virtualMail; + period = "*:30"; + }; + } + ''; + + description = '' + Duplicity backup configurations. Each attribute names a backup + to be created at a given time interval, according to the options + associated with it. + + For each member of the set is created a timer which triggers the + instanced duplicity-backup-name service unit. You may use + systemctl start duplicity-backup-name to + manually trigger creation of backup-name at + any time. + ''; + }; + }; + }; + + config = mkIf gcfg.enable { + assertions = concatLists (mapAttrsToList (name: cfg: + let + protocols = [ "s3" "sftp" ]; + in + [{ assertion = any (protocol: hasPrefix (protocol + "://") cfg.destination) protocols; + message = "Currently supported protocols are: " + concatStringsSep " " protocols; + } + ]) gcfg.archives); + + environment.systemPackages = [ duplicityGenKeys ]; + }; +} diff --git a/duplicity-backup-restore.nix b/duplicity-backup-restore.nix new file mode 100644 index 0000000..0132e72 --- /dev/null +++ b/duplicity-backup-restore.nix @@ -0,0 +1,80 @@ +{ config, lib, pkgs, utils, ... }: + +with lib; + +let + gcfg = config.services.duplicity-backup; + + mkSecurePathOption = + { description + , default + }: + mkOption { + inherit description default; + + type = with types; either path string; + apply = x: if builtins.typeOf x == "path" then toString x else x; + }; +in +{ + imports = [ ./duplicity-backup-common.nix ]; + + options = { + services.duplicity-backup.enableRestore = mkEnableOption "periodic duplicity backups restore tools"; + + services.duplicity-backup.target = mkSecurePathOption { + description = '' + The restoration target directory, + useful for restoring a target directory to /mnt + ''; + default = ""; + }; + + services.duplicity-backup.archives = mkOption { + type = types.attrsOf (types.submodule ({ name, config, ... }: + { + options = { + target = mkSecurePathOption { + description = '' + The restoration target directory, + useful for restoring a target directory to /mnt + ''; + default = gcfg.target; + }; + + enableForce = mkOption { + type = types.bool; + default = false; + }; + + script = mkOption { + type = types.path; + readOnly = true; + }; + }; + + config.script = pkgs.writeScriptBin "duplicity-restore-${name}" ('' + for i in ${gcfg.envDir}/*; do + source $i + done + + mkdir -p ${config.target + dirOf config.directory} + + ${pkgs.duplicity}/bin/duplicity \ + --archive-dir ${gcfg.cacheDir} \ + --name ${name} \ + --gpg-options "--homedir=${gcfg.pgpDir}" \ + '' + optionalString (config.enableForce) ''--force \ + '' + optionalString (!gcfg.usePassphrase) ''--encrypt-key "Duplicity Backup" \'' + '' + ${config.destination} \ + ${config.target + config.directory} + ''); + } + )); + }; + }; + + config = mkIf (gcfg.enable && gcfg.enableRestore) { + environment.systemPackages = mapAttrsToList (name: value: value.script) gcfg.archives; + }; +} diff --git a/duplicity-backup.nix b/duplicity-backup.nix index a03e64d..fb2fe44 100644 --- a/duplicity-backup.nix +++ b/duplicity-backup.nix @@ -2,296 +2,8 @@ with lib; -let - gcfg = config.services.duplicity-backup; -in { - options = { - services.duplicity-backup = { - enable = mkEnableOption "periodic duplicity backups"; - - sshIdentityFile = mkOption { - type = types.path; - default = /var/empty; - description = '' - ''; - }; - - knownHostsFile = mkOption { - type = types.path; - default = /var/empty; - description = '' - ''; - }; - - keyring = mkOption { - type = types.path; - default = /var/empty; - description = '' - ''; - }; - - keyId = mkOption { - type = types.string; - default = "00000000"; - description = '' - ''; - }; - - pgpKeyFile = mkOption { - type = types.path; - default = /var/empty; - description = '' - ''; - }; - - passphraseFile = mkOption { - type = types.path; - default = /var/empty; - description = '' - ''; - }; - - archives = mkOption { - type = types.attrsOf (types.submodule ({ config, ... }: - { - options = { - - requiredNixopsKeys = mkOption { - type = types.listOf types.string; - default = [ ]; - example = [ "my-passphrase" "my-ssh-id" ]; - description = '' - A list of nixops keys on which to depend - (will create the necessary -key.service - systemd dependencies) - ''; - }; - - sshIdentityFile = mkOption { - type = types.path; - default = gcfg.sshIdentityFile; - description = '' - Set a specific ___ for this archive. This defaults to - if left unspecified. - ''; - }; - - knownHostsFile = mkOption { - type = types.path; - default = gcfg.knownHostsFile; - description = '' - Set a specific ___ for this archive. This defaults to - if left unspecified. - ''; - }; - - keyring = mkOption { - type = types.path; - default = gcfg.keyring; - description = '' - Set a specific ___ for this archive. This defaults to - if left unspecified. - ''; - }; - - pgpKeyFile = mkOption { - type = types.path; - default = gcfg.gpgKeyFile; - description = '' - Set a specific ___ for this archive. This defaults to - if left unspecified. - ''; - }; - - passphraseFile = mkOption { - type = types.path; - default = gcfg.passphraseFile; - description = '' - Set a specific ___ for this archive. This defaults to - if left unspecified. - ''; - }; - - keyId = mkOption { - type = types.string; - default = gcfg.keyId; - description = '' - Set a specific ___ for this archive. This defaults to - if left unspecified. - ''; - }; - - cachedir = mkOption { - type = types.path; - default = "/var/cache/duplicity/"; - description = '' - The cache allows duplicity to identify previously stored data - blocks, reducing archival time and bandwidth usage. - ''; - }; - - destination = mkOption { - type = types.string; - default = ""; - example = "rsync://user@example.com:/home/user"; - description = '' - ''; - }; - - period = mkOption { - type = types.str; - default = "01:15"; - example = "hourly"; - description = '' - Create archive at this interval. - - The format is described in - systemd.time - 7. - ''; - }; - - directories = mkOption { - type = types.listOf types.path; - default = []; - description = "List of filesystem paths to archive."; - }; - - excludes = mkOption { - type = types.listOf types.str; - default = []; - description = '' - Exclude files and directories matching these patterns. - ''; - }; - - includes = mkOption { - type = types.listOf types.str; - default = []; - description = '' - Include only files and directories matching these - patterns (the empty list includes everything). - - Exclusions have precedence over inclusions. - ''; - }; - - maxbw = mkOption { - type = types.nullOr types.int; - default = null; - description = '' - Abort archival if upstream bandwidth usage in bytes - exceeds this threshold. - ''; - }; - - maxbwRateUp = mkOption { - type = types.nullOr types.int; - default = null; - example = literalExample "25 * 1000"; - description = '' - Upload bandwidth rate limit in bytes. - ''; - }; - - maxbwRateDown = mkOption { - type = types.nullOr types.int; - default = null; - example = literalExample "50 * 1000"; - description = '' - Download bandwidth rate limit in bytes. - ''; - }; - }; - } - )); - - default = {}; - - example = literalExample '' - { - nixos = - { directories = [ "/home" "/root/ssl" ]; - }; - - gamedata = - { directories = [ "/var/lib/virtualMail" ]; - period = "*:30"; - }; - } - ''; - - description = '' - Duplicity backup configurations. Each attribute names a backup - to be created at a given time interval, according to the options - associated with it. - - For each member of the set is created a timer which triggers the - instanced duplicity-backup-name service unit. You may use - systemctl start duplicity-backup-name to - manually trigger creation of backup-name at - any time. - ''; - }; - }; - }; - - config = mkIf gcfg.enable { - assertions = - (mapAttrsToList (name: cfg: - { assertion = cfg.directories != []; - message = "Must specify paths for duplicity to back up"; - }) gcfg.archives); - - systemd.services = - mapAttrs' (name: cfg: nameValuePair "duplicity-${name}" { - description = "Duplicity archive '${name}'"; - requires = [ "network-online.target" ] - ++ (map (k: k + "-key.service") cfg.requiredNixopsKeys); - after = [ "network-online.target" ] - ++ (map (k: k + "-key.service") cfg.requiredNixopsKeys); - - path = with pkgs; [ iputils duplicity openssh gnupg utillinux ]; - - # make sure that the backup server is reachable - #preStart = '' - # while ! ping -q -c 1 ${findawaytoextracttheaddressmaybe} &> /dev/null; do sleep 3; done - #''; - - script = '' - mkdir -p ${cfg.cachedir} - chmod 0700 ${cfg.cachedir} - gpg --import ${cfg.pgpKeyFile} # FIXME - export PASSPHRASE=$(cat ${cfg.passphraseFile}) - duplicity \ - --archive-dir ${cfg.cachedir} \ - --name ${name} \ - --ssh-options "-i '${cfg.sshIdentityFile}' -oUserKnownHostsFile='${cfg.knownHostsFile}'" \ - # --gpg-options "--no-default-keyring --keyring ${cfg.keyring}" \ - --encrypt-sign-key ${cfg.keyId} \ - ${concatStringsSep " " (map (v: "--exclude ${v}") cfg.excludes)} \ - ${concatStringsSep " " (map (v: "--include ${v}") cfg.includes)} \ - ${concatStringsSep " " cfg.directories} \ - ${cfg.destination} - ''; - - serviceConfig = { - Type = "oneshot"; - IOSchedulingClass = "idle"; - NoNewPrivileges = "true"; - CapabilityBoundingSet = [ "CAP_DAC_READ_SEARCH" ]; - PermissionsStartOnly = "true"; - }; - }) gcfg.archives; - - # Note: the timer must be Persistent=true, so that systemd will start it even - # if e.g. your laptop was asleep while the latest interval occurred. - systemd.timers = mapAttrs' (name: cfg: nameValuePair "duplicity-${name}" - { timerConfig.OnCalendar = cfg.period; - timerConfig.Persistent = "true"; - wantedBy = [ "timers.target" ]; - }) gcfg.archives; - - }; + imports = [ ./duplicity-backup-common.nix + ./duplicity-backup-backup.nix + ./duplicity-backup-restore.nix ]; } diff --git a/duplicity-system.nix b/duplicity-system.nix new file mode 100644 index 0000000..e803e48 --- /dev/null +++ b/duplicity-system.nix @@ -0,0 +1,80 @@ +{ pkgs +, lib +, options +, config +, ... +}: + +with lib; +let + dcfg = config.services.duplicity-backup; + cfg = config.services.duplicity-system; + + missing_imports = let + inherit (config.services.duplicity-backup.archives.system) includes; + in lib.unique + (lib.filter (file: !(lib.any (pfx: lib.hasPrefix (toString pfx) file) ([ ] ++ includes) || + lib.any (sfx: lib.hasSuffix (toString sfx) file) [ "" "system-specific.nix" "hardware-configuration.nix" ])) + (builtins.map (x: toString x.file) options._definedNames)); +in +{ + imports = [ ./duplicity-backup.nix ]; + + options.services.duplicity-system = { + restorationImage = lib.mkOption { + type = with types; bool; + default = false; + }; + + destination = lib.mkOption { + type = with types; string; + }; + + includes = lib.mkOption { + type = with types; listOf path; + default = []; + }; + + extraExcludes = lib.mkOption { + type = with types; listOf path; + }; + }; + + config = { + # assertions = + # [ { assertion = !cfg.restorationImage -> missing_imports == []; + # message = "Missing imports in system archive: ${toString missing_imports}"; + # } + # ]; + + services.duplicity-backup = { + enable = true; + usePassphrase = true; + + archives.system = { + inherit (cfg) destination; + allowSourceMismatch = true; + + directory = /.; + + includes = cfg.includes; + + excludes = [ + /boot + /dev + /lost+found + /nix + /proc + /run + /sys + /tmp + /usr + /etc/nixos/system-specific.nix + /etc/nixos/hardware-configuration.nix + dcfg.cacheDir + dcfg.envDir + ] ++ cfg.extraExcludes; + }; + }; + }; +}