Skip to content

Add a test suite and run it in CI - #11

Merged
didvc merged 1 commit into
masterfrom
tests
Sep 19, 2026
Merged

didvc merged 1 commit into
masterfrom
tests

Conversation

@didvc

@didvc didvc commented Sep 19, 2026

Copy link
Copy Markdown
Owner

The project had no tests for what it actually promises. shellcheck ran on every push, but nothing checked that the sandbox still isolates anything — the failure mode that doesn't announce itself, since a wrapper that starts fine and confines nothing looks exactly like a working one.

84 tests in seven tiers, plain bash, no dependencies beyond git and coreutils. CONTRIBUTING.md says dependency-free and lightweight, so there's no bats or similar; tiers that need a sandbox skip rather than fail where bwrap is missing, so tests/run.sh is runnable anywhere.

tier covers
10-core namespace and mount invariants: unshare-all, die-with-parent, ro /usr and /etc, private /tmp, HOME as an empty dir, mount ordering
20-agents registry, aliases, per-agent passthrough, and that no agent widens the sandbox to ~, ~/.ssh or ~/.gnupg
30-options parsing, missing values, passthrough, -- handling
40-git which git dir is mounted read-only, and when
50-config global drop-ins, per-user file, precedence
60-confinement the real sandbox: what's writable, what's reachable, network namespacing
70-overlay the write-back path — the only code here that deletes the user's files

CI gains two jobs: one installs the distro bubblewrap and runs what it can; the other builds bubblewrap 0.11 from source, since --overlay needs it and the packaged version is older. That job fails loudly if the built binary lacks the flag, rather than letting the tier skip into a green run that tested nothing.

A bug the suite found. From a subdirectory, older git returns --git-common-dir relative and with .. components, so prefixing $WORK_DIR produced a bind source like /repo/sub/deeper/../../.git. That works only by luck: .. is applied after symlinks, so with a symlinked path component it can name a different directory than the repository's own git dir — in the code deciding what stays read-only. Now canonicalised.

Mutation-checked, so the tests aren't vacuous:

deliberate break result
drop --unshare-all 3 failures
make /usr writable 2 failures
invert the .git read-only default 6 failures

Known gap: the opaque-directory case can't be exercised inside a nested user namespace — the kernel refuses the marker with EIO. Those tests skip there and run on a normal host, which is one reason the overlay job exists in CI.

The project had no tests for what it actually promises. shellcheck ran on
every push, but nothing checked that the sandbox still isolates anything,
which is the failure mode that does not announce itself: a wrapper that
starts fine and confines nothing looks exactly like a working one.

84 tests in seven tiers, plain bash, no dependencies beyond git and
coreutils. Tiers that need a sandbox skip rather than fail where bwrap is
missing or user namespaces are unavailable, so tests/run.sh is runnable
anywhere:

  10-core         namespace and mount invariants (unshare-all,
                  die-with-parent, ro /usr and /etc, private /tmp, HOME
                  is an empty dir and not the real one, mount ordering)
  20-agents       registry, aliases, per-agent passthrough, and that no
                  agent widens the sandbox to ~, ~/.ssh or ~/.gnupg
  30-options      parsing, missing values, passthrough, -- handling
  40-git          which git dir is mounted read-only, and when
  50-config       global drop-ins, per-user file, precedence
  60-confinement  the real sandbox: what is writable, what is reachable,
                  network namespacing
  70-overlay      the write-back path, the only code here that deletes
                  the user's files

CI gains two jobs. One installs the distro bubblewrap and runs everything
it can. The other builds bubblewrap 0.11 from source, because --overlay
needs it and the distro package is older, and fails loudly if the built
binary lacks the flag rather than letting the tier skip into a green run
that tested nothing.

Also fixes a path bug the suite found. From a subdirectory, older git
returns --git-common-dir relative and with .. components, and prefixing
$WORK_DIR left a bind source like /repo/sub/deeper/../../.git. That
resolves correctly only by luck: .. is applied after symlinks, so with a
symlinked path component it can name a different directory than the
repository's own git dir, in the code that decides what stays read-only.
Canonicalised instead.

Known gap: the opaque-directory case cannot be exercised inside a nested
user namespace, where the kernel refuses the marker with EIO. Those tests
skip there and run on a normal host.
@didvc
didvc merged commit 6fcf0af into master Sep 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant