Skip to content

Commit ed7ae62

Browse files
ThomasWaldmannThomasWaldmann
authored
Merge pull request #10005 from ThomasWaldmann/repo-compress-9663
re-add repo-compress, fixes #9663
2 parents 3325449 + 637972f commit ed7ae62

10 files changed

Lines changed: 806 additions & 67 deletions

File tree

‎docs/faq.rst‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,8 @@ are calculated *before* compression. New compression settings
464464
will only be applied to new chunks, not existing chunks. So it's safe
465465
to change them.
466466

467+
Use ``borg repo-compress`` to efficiently recompress a complete repository.
468+
467469
Why is backing up an unmodified FAT filesystem slow on Linux?
468470
-------------------------------------------------------------
469471

‎docs/usage.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@ Usage
4040
usage/repo-space
4141
usage/repo-list
4242
usage/repo-info
43+
usage/repo-compress
4344
usage/repo-delete
4445
usage/serve
4546
usage/version

‎docs/usage/repo-compress.rst‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.. include:: repo-compress.rst.inc
2+
3+
Examples
4+
~~~~~~~~
5+
6+
::
7+
8+
# Recompress repository contents
9+
$ borg repo-compress --progress --compression=zstd,3
10+
11+
# Recompress and obfuscate repository contents
12+
$ borg repo-compress --progress --compression=obfuscate,1,zstd,3

‎docs/usage/repo-compress.rst.inc‎

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
.. IMPORTANT: this file is auto-generated from borg's built-in help, do not edit!
2+
3+
.. _borg_repo-compress:
4+
5+
borg repo-compress
6+
------------------
7+
.. code-block:: none
8+
9+
borg [common options] repo-compress [options]
10+
11+
.. only:: html
12+
13+
.. class:: borg-options-table
14+
15+
+-------------------------------------------------------+---------------------------------------------------+--------------------------------------------------------------------------------------------------+
16+
| **options** |
17+
+-------------------------------------------------------+---------------------------------------------------+--------------------------------------------------------------------------------------------------+
18+
| | ``-C COMPRESSION``, ``--compression COMPRESSION`` | select compression algorithm, see the output of the "borg help compression" command for details. |
19+
+-------------------------------------------------------+---------------------------------------------------+--------------------------------------------------------------------------------------------------+
20+
| | ``-s``, ``--stats`` | print statistics |
21+
+-------------------------------------------------------+---------------------------------------------------+--------------------------------------------------------------------------------------------------+
22+
| .. class:: borg-common-opt-ref |
23+
| |
24+
| :ref:`common_options` |
25+
+-------------------------------------------------------+---------------------------------------------------+--------------------------------------------------------------------------------------------------+
26+
27+
.. raw:: html
28+
29+
<script type='text/javascript'>
30+
$(document).ready(function () {
31+
$('.borg-options-table colgroup').remove();
32+
})
33+
</script>
34+
35+
.. only:: latex
36+
37+
38+
39+
options
40+
-C COMPRESSION, --compression COMPRESSION select compression algorithm, see the output of the "borg help compression" command for details.
41+
-s, --stats print statistics
42+
43+
44+
:ref:`common_options`
45+
|
46+
47+
Description
48+
~~~~~~~~~~~
49+
50+
Repository (re-)compression (and/or re-obfuscation).
51+
52+
Reads all repository objects and recompresses the ones that are not already using
53+
the compression type/level and obfuscation level given via ``--compression``.
54+
55+
The repository is processed one pack file at a time: a pack is read as a whole and,
56+
if it holds objects that need recompression, rewritten as a whole - objects already
57+
using the desired compression are copied into the rewritten pack unchanged. A pack
58+
whose objects all already use the desired compression is not touched at all.
59+
Please note that the outcome of recompressing a chunk might not always be the
60+
desired compression type/level - if no compression gives a shorter output, that
61+
might be chosen; such chunks are kept as they are.
62+
63+
Rewriting a pack invalidates every client's cached chunk index, so the next borg
64+
operation of each client will re-fetch the chunk index from the repository.
65+
66+
This command needs free space in the repository for the rewritten pack files
67+
(roughly one pack file size while running).
68+
69+
If the ``borg repo-compress`` process receives a SIGINT signal (Ctrl-C), it stops
70+
at the next pack boundary, leaving the repository and its chunk index in a
71+
consistent state; running it again later processes the remaining packs.
72+
73+
Both ``--progress`` and ``--stats`` are recommended when ``borg repo-compress``
74+
is used interactively.
75+
76+
You do **not** need to run ``borg compact`` after ``borg repo-compress``.

‎scripts/shell_completions/fish/borg.fish‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ complete -c borg -f -n __fish_is_first_token -a 'compact' -d 'Free repository sp
2020
complete -c borg -f -n __fish_is_first_token -a 'info' -d 'Show archive details'
2121
complete -c borg -f -n __fish_is_first_token -a 'mount' -d 'Mount archive or a repository'
2222
complete -c borg -f -n __fish_is_first_token -a 'umount' -d 'Unmount the mounted archive'
23+
complete -c borg -f -n __fish_is_first_token -a 'repo-compress' -d 'Repository (re-)compression'
2324
complete -c borg -f -n __fish_is_first_token -a 'repo-create' -d 'Create a new, empty repository'
2425
complete -c borg -f -n __fish_is_first_token -a 'repo-delete' -d 'Delete a repository'
2526
complete -c borg -f -n __fish_is_first_token -a 'repo-info' -d 'Show repository information'
@@ -98,6 +99,12 @@ complete -c borg -f -l 'newest' -d 'Consider archives within
9899
complete -c borg -f -l 'older' -d 'Consider archives older than TIMESPAN' -n "__fish_seen_subcommand_from analyze"
99100
complete -c borg -f -l 'newer' -d 'Consider archives newer than TIMESPAN' -n "__fish_seen_subcommand_from analyze"
100101

102+
# borg repo-compress options
103+
# Define compression methods once at the top
104+
set -l compression_methods "none auto lz4 zstd,1 zstd,2 zstd,3 zstd,4 zstd,5 zstd,6 zstd,7 zstd,8 zstd,9 zstd,10 zstd,11 zstd,12 zstd,13 zstd,14 zstd,15 zstd,16 zstd,17 zstd,18 zstd,19 zstd,20 zstd,21 zstd,22 zlib,1 zlib,2 zlib,3 zlib,4 zlib,5 zlib,6 zlib,7 zlib,8 zlib,9 lzma,0 lzma,1 lzma,2 lzma,3 lzma,4 lzma,5 lzma,6 lzma,7 lzma,8 lzma,9"
105+
complete -c borg -f -s C -l 'compression' -d 'Select compression ALGORITHM,LEVEL [lz4]' -a "$compression_methods" -n "__fish_seen_subcommand_from repo-compress"
106+
complete -c borg -f -s s -l 'stats' -d 'Print statistics' -n "__fish_seen_subcommand_from repo-compress"
107+
101108
# borg create options
102109
complete -c borg -f -s n -l 'dry-run' -d 'Do not create a backup archive' -n "__fish_seen_subcommand_from create"
103110
complete -c borg -f -s s -l 'stats' -d 'Print verbose statistics' -n "__fish_seen_subcommand_from create"

‎src/borg/archiver/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@
8181
from .lock_cmds import LocksMixIn
8282
from .mount_cmds import MountMixIn
8383
from .prune_cmd import PruneMixIn
84+
from .repo_compress_cmd import RepoCompressMixIn
8485
from .recreate_cmd import RecreateMixIn
8586
from .rename_cmd import RenameMixIn
8687
from .repo_create_cmd import RepoCreateMixIn
@@ -117,6 +118,7 @@ class Archiver(
117118
PruneMixIn,
118119
RecreateMixIn,
119120
RenameMixIn,
121+
RepoCompressMixIn,
120122
RepoCreateMixIn,
121123
RepoDeleteMixIn,
122124
RepoInfoMixIn,
@@ -305,6 +307,7 @@ def build_parser(self):
305307
self.build_parser_locks(subparsers, common_parser, mid_common_parser)
306308
self.build_parser_mount_umount(subparsers, common_parser, mid_common_parser)
307309
self.build_parser_prune(subparsers, common_parser, mid_common_parser)
310+
self.build_parser_repo_compress(subparsers, common_parser, mid_common_parser)
308311
self.build_parser_repo_create(subparsers, common_parser, mid_common_parser)
309312
self.build_parser_repo_delete(subparsers, common_parser, mid_common_parser)
310313
self.build_parser_repo_info(subparsers, common_parser, mid_common_parser)

0 commit comments

Comments
 (0)