Skip to content
Draft

Fixes #779

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 30 additions & 4 deletions .github/workflows/build-and-test.yml
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
name: Build and test (develop)
name: Build and test

on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches:
- master
- develop

jobs:
Expand All @@ -14,31 +15,54 @@ jobs:
RM_TS_DIR: "/tmp/rmlint-unit-testdir"
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: "Prepare build environment"
run: |
sudo apt update
sudo apt install -y --no-install-recommends \
sudo apt-get update
sudo apt-get install -y --no-install-recommends \
scons python3-sphinx gettext python3-setuptools \
libblkid-dev libelf-dev libglib2.0-dev libjson-glib-dev \
clang python3-pip python3-cffi python3-dev libffi-dev
pip3 install -r tests/requirements.txt
pip3 install -r docs/requirements.txt

- name: "Build"
# Todo: eventually run tests with valgrind (RM_TS_USE_VALGRIND)
# Todo enable slow tests in pytest
run: |
scons config
scons VERBOSE=1 DEBUG=1 O=release

- name: "Check need for testing"
# TODO also include skipping build but allow docs to be processed
run: |
if git diff --exit-code --name-only origin/$GITHUB_BASE_REF...$GITHUB_SHA ':!.gitignore' ':!docs' ':!*.md' ':!*.txt' ':!*.rst'; then
echo "RUN_TEST=false" >> "$GITHUB_ENV"
echo "RUN_TEST = false"
else
echo "RUN_TEST=true" >> "$GITHUB_ENV"
echo "RUN_TEST = true"
fi

- name: "Prepare test environment"
# TODO Ignore linter warning. See https://github.com/github/vscode-github-actions/issues/222
if: ${{ env.RUN_TEST == 'true' }}
# The test suite is seriously disk-intensive. Given that linux
# instances hosted in GitHub have 16G of RAM available we will
# use it to speed up the run.
run: |
sudo mkdir "${RM_TS_DIR}"
sudo mount -o size=12G,nr_inodes=0 -t tmpfs tmpfs "${RM_TS_DIR}"
- name: "Test"

- name: "Test it"
if: ${{ env.RUN_TEST == 'true' }}
run: |
RM_TS_PRINT_CMD=0 RM_TS_PEDANTIC=0 python -m pytest -s -v

- name: CoW tests
if: ${{ env.RUN_TEST == 'true' }}
shell: bash
run: |
sudo umount "${RM_TS_DIR}"
Expand All @@ -61,7 +85,9 @@ jobs:
items[:] = selected_items
EOF
RM_TS_PRINT_CMD=0 RM_TS_PEDANTIC=0 python -m pytest -s -v

- name: "Cleanup"
if: ${{ env.RUN_TEST == 'true' }}
run: |
sudo umount "${RM_TS_DIR}"
sudo rmdir "${RM_TS_DIR}"
136 changes: 136 additions & 0 deletions .github/workflows/multi-distro-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
name: Multi-distro build

on:
pull_request:
types: [opened, synchronize, reopened]
push:
branches:
- master

jobs:
multi-distro-build:
runs-on: ubuntu-latest
if: always()
strategy:
matrix:
build:
#############
# AlmaLinux #
#############
- {
tag: "almalinux:9",
installcmd: "dnf install -y epel-release && dnf install -y",
pkgs: "glib2-devel json-glib-devel gcc python3-scons",
}
- {
tag: "almalinux:10-kitten",
installcmd: "dnf install -y epel-release && dnf install -y",
pkgs: "glib2-devel json-glib-devel gcc python3-scons",
}
##########
# Alpine #
##########
- {
tag: "alpine:latest",
installcmd: "apk add",
pkgs: "build-base glib-dev json-glib-dev pkgconf scons",
}
#############
# ArchLinux #
#############
- {
tag: "archlinux:latest",
installcmd: "pacman -Sy --noconfirm",
pkgs: "base-devel git json-glib scons",
}
###########
# CachyOS #
###########
- {
tag: "cachyos/cachyos:latest",
installcmd: "pacman -Sy --noconfirm",
pkgs: "base-devel git json-glib scons",
}
##########
# Debian #
##########
- {
tag: "debian:bullseye",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
- {
tag: "debian:bookworm",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
- {
tag: "debian:trixie",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
##########
# Fedora #
##########
- {
tag: "fedora:latest",
installcmd: "dnf install -y",
pkgs: "glib2-devel json-glib-devel gcc python3-scons",
}
############
# OpenSUSE #
############
- {
tag: "opensuse/leap:latest",
installcmd: "zypper -n install",
pkgs: "json-glib-devel glib2-devel patterns-devel-base-devel_basis scons",
}
- {
tag: "opensuse/tumbleweed:latest",
installcmd: "zypper -n install",
pkgs: "json-glib-devel glib2-devel patterns-devel-base-devel_basis scons",
}
##########
# Ubuntu #
##########
- {
tag: "ubuntu:20.04",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
- {
tag: "ubuntu:22.04",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
- {
tag: "ubuntu:24.04",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
- {
tag: "ubuntu:rolling",
env: "DEBIAN_FRONTEND=noninteractive",
installcmd: "apt update && apt install -y --no-install-recommends",
pkgs: "clang gettext libblkid-dev libelf-dev libffi-dev libglib2.0-dev libjson-glib-dev python3-cffi python3-dev python3-pip python3-setuptools python3-sphinx scons",
}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: "Build on ${{ matrix.build.tag }}"
run: >
echo "
export UNUSED_VAR_SO_THIS_CMD_IS_NEVER_EMPTY= ${{ matrix.build.env }}
${{ matrix.build.installcmd }} ${{ matrix.build.pkgs }}
cd /build
scons config
scons VERBOSE=1 DEBUG=1 O=release
exit
" | docker run --rm -i -v "$PWD:/build" "${{ matrix.build.tag }}" sh
11 changes: 8 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ docs/rmlint.1
.sconf*
.rope*
*.pyc
/.prefix.txt

docs/_build
__pycache__
Expand All @@ -25,9 +26,13 @@ lib/config.h
lib/formats/py.c
lib/formats/sh.c
uninstall-
gui/app/resources/app.gresource
*.a

rmlint.*.sh
rmlint.*.json
/rmlint.*.sh
/rmlint.*.json
/.rmlint.json

/rmlint-*.tar.gz

# JetBrains' IDE
.idea
10 changes: 10 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
version: 2
build:
os: ubuntu-22.04
tools:
python: "3.12"
sphinx:
configuration: docs/conf.py
python:
install:
- requirements: docs/requirements.txt
2 changes: 1 addition & 1 deletion .version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.10.1 Ludicrous Lemur
2.10.3 Ludicrous Lemur
74 changes: 73 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ The format follows [keepachangelog.com]. Please stick to it.
* Can now atomically clone from original to its hardlink via ``rmlint --dedupe``
* Option ``-c json:traversed`` to include list of fully-traversed dirs in json output
* Option ``--ignore-bad-paths`` to not abort run if one or more bad paths passed
* Exit code 12 for ``rmlint --is-reflink``: inline extents.

### Changed

Expand All @@ -39,12 +40,83 @@ The format follows [keepachangelog.com]. Please stick to it.
* Small files (<4k) are now generally ignored for reflinking / cloning options since They
are typically stored as inline extents
* docker: update to a more recent alpine and install full test dependencies
* short option ``-U`` now refers to ``--hash-unmatched`` instead of ``--write-unfinished``.

### Removed
* ``--write-unfinished``. Use ``--hash-unmatched``.
* ``--dedupe-xattr``. Use ``--dedupe --xattr``
* ``--dedupe-readonly``. Use ``--dedupe --readonly``

## [2.10.3 Ludicrous Lemur] - 2025-03-22

### Added
- Option `--without-compile-glib-schemas` to _scons_ to avoid (re)compiling system schemas on (un)install (#710).
- Compatibility with Solaris (#533).
- Compatibility with older macOS for PowerPC support (#573).
- Github CI (#675, #677, #699, #703, #718).

### Changed
- GUI: migration of GtkSourceView from 3.0 to 4.0 (#719).
- Glib: unify Glib minimum version requirement to 2.64 for all architectures (#707, #716).
- Documentation updates (#538, #560, #565, #568, #585, #591, #678, #636, #643, #693, #695, #714, #715, #722).
- Use compiler from `CC` environ if available (#526).
- Manpage is not longer compressed (#525).
- Stampfiles are created in /tmp instead of the current directory (#697).
- The same stampfiles are reused throughout the whole run (#697).
- Translations (#720).

### Fixed
- Fix string format in tests (#602).
- Fix configuration checks of xattr-functions (#605).
- Fix Python warnings during configuration (#684).
- Fix recheck in generated shell-script (#683).
- Fix data-loss issue when using replays that does not contains checksums (e.g. with hardlinks) (#689).
- Avoid generating an empty '-.o' file during configuration (#686).
- Escape CWD and args strings in generated JSON output (#691).
- Stampfiles are no longer created on dry-runs (#697).
- Stampfiles are removed on exit (#697).
- Compilation warnings with GLib >= 2.84 (#708).
- Shredder bootstrapping (#709).
- Code linting (#639, #701).

## [2.10.2 Ludicrous Lemur] -- 2023-08-08

### Changed

* preserve coredump and exit status for fatal signals
* --equal no longer writes rmlint.sh and rmlint.json by default
* issue #458: --is-reflink no longer accepts the '-o' option or writes rmlint.sh/rmlint.json
* issue #434: downgraded 'Added big file "\<filename ...\>' from warning to info

### Fixed

* PR #443: scons: make fiemap feature depend on --with-fiemap, not --with-gettext
* PR #470: scons: read environment variables before conftests
* PR #474: scons: when printing parameters such as DEBUG, interpret '0' as 'no'
* PR #485, issue #484: fix sorting of regex options (-S x and -S r) when multiple paths match
* issue #496: fix 'free(): invalid pointer' crash with `-c csv:unique`
* issue #438: fix 'Can't open directory or file "...": Invalid argument' on some platforms
* issue #621: fix GUI freeze with glib 2.75.3 and above
* issue #608: fix setuptools InvalidVersion error when installing GUI with packaging 22.0 and above
* issue #613: actually remove the GUI's Polkit requirement ('Namespace Polkit not available')
* issue #522: fix --size overflow detection on some platforms
* issue #549: fix 'assertion failed: (node-\>inode != RM\_NO\_INODE)' on 32-bit platforms
* issue #555: fix deadlock when `-T nonstripped` encountered an executable FIFO
* issues #527, #528, #529, #530, #611: fix --is-reflink false-positives and false-negatives
* fix --merge-directories false-negatives with `--algorithm sha1`
* issue #577: fix GUI error 'Unknown option --match-with-extension'
* issue #552: prevent generated script's -p option from overwriting itself with --merge-directories
* issue #475: fix 'assertion failed: (self-\>num\_pending == 0)' with --merge-directories
* issue #562: fix 'double free or corruption (out)' with --merge-directories --write-unfinished
* issue #545: fix skip\_hardlink false-negatives with --merge-directories
* do not write checksums to xattrs when --clamp-low/--clamp-top are in use
* issue #431: suppress 'failed to getxattr for ...: Attribute not found' on macOS
* fix progress text not displaying final results before continuing to next stage
* fix regular files and symlinks potentially matching with --see-symlinks
* fix assertion failure with --merge-directories and zero paths from stdin
* issue #519: suppress 'Permission denied' errors from xfs\_info
* fix --hash for large files on 32-bit platforms
* issue #628: fix 'SyntaxError: source code cannot contain null bytes' with --gui and Python 3.12

## [2.10.1 Ludicrous Lemur] -- 2020-06-13

### Added
Expand Down
Loading
Loading