Skip to content

feat(e2e): compose test software as a container layer instead of runtime install - #744

Merged
castrojo merged 1 commit into
mainfrom
claude/e2e-compose-image
Aug 25, 2026
Merged

feat(e2e): compose test software as a container layer instead of runtime install#744
castrojo merged 1 commit into
mainfrom
claude/e2e-compose-image

Conversation

@hanthor

@hanthor hanthor commented Aug 8, 2026

Copy link
Copy Markdown
Member

Motivation

rpm-ostree install --apply-live fails on images that deliberately lock runtime layering: bluefin-lts ships rpm-ostreed.conf with LockLayering=true (projectbluefin/bluefin-lts#492, shipped in projectbluefin/bluefin-lts#509). The e2e harness currently layers the common suite's shell tools at runtime inside the booted VM, which breaks against such images. Per maintainer direction, the harness now composes the test software as a container layer at build time — following the pr-e2e.yml pattern in bluefin-lts (Containerfile FROM base + RUN, pushed to GHCR with a run-scoped tag).

Compose flow

  1. New compose job in the reusable e2e.yml (runs after the matrix resolve, before the suite jobs; packages: write):
    • checks out container/e2e-overlay/ + scripts/compose-e2e-image.sh at inputs.test_ref
    • scripts/compose-e2e-image.sh builds container/e2e-overlay/Containerfile with --build-arg BASE_IMAGE=<inputs.image> and pushes ghcr.io/<owner>/testsuite-e2e:run-${GITHUB_RUN_ID}
    • exports the composed ref as a job output
  2. Suite jobs boot needs.compose.outputs.image || inputs.image (GHCR login added before the pull, since fresh run-scoped packages default to private).
  3. The overlay installs the tools the common suite validates — zsh fish fzf bat eza fd-find ripgrep starship (container/e2e-overlay/packages.list) — best-effort per package, mirroring the old step's warning-on-missing tolerance, and runs ostree container commit when applicable.
  4. Summary and screenshot naming (IMAGE_SLUG, gh-pages URL, artifact metadata) keep using the raw input image so run-scoped tags don't leak into artifact names.

Fallback toggle

  • compose_image workflow input (default true), or repository variable E2E_COMPOSE_IMAGE=0, disables composing.
  • A compose-job failure is non-fatal (continue-on-error + warning): suite jobs automatically fall back to the raw image and the legacy runtime-install path.

Runtime-install sites replaced / kept

  • Replaced: e2e.yml "Install shell tools for common suite" — the sudo rpm-ostree install --apply-live / dnf install calls inside the VM (the feat(common): add Bluefin-specific GSettings override validation #492 offender). Kept only as the legacy fallback step; on the composed path it becomes a verification-only step.
  • Deliberately left on the old path:
    • scripts/install-kde-webdriver.sh (KDE suites) — dnf/source-build inside the VM; ABI-tied to the SUT per its own documented rationale, out of scope here.
    • tests/dx distrobox dnf install — runs inside a mutable container, unaffected by LockLayering.
    • Runner-container microdnf patching and Flatpak preloads — container/Flatpak operations, not host layering.

🤖 Generated with Claude Code

https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc


Generated by Claude Code

…ime install

rpm-ostree install --apply-live fails on images that ship
rpm-ostreed.conf LockLayering=true (bluefin-lts does this deliberately —
projectbluefin/bluefin-lts#492, PR #509), so the harness can no longer
rely on installing the common suite's shell tools at runtime inside the
VM.

Add a compose stage to the reusable e2e workflow:

- container/e2e-overlay/Containerfile + packages.list: derived layer
  FROM the target image that best-effort installs the tools the common
  suite validates (zsh, fish, fzf, bat, eza, fd-find, ripgrep, starship).
- scripts/compose-e2e-image.sh: builds the overlay with podman and
  pushes it to GHCR.
- e2e.yml `compose` job: pushes
  ghcr.io/<owner>/testsuite-e2e:run-<GITHUB_RUN_ID> and exports the
  composed ref; suite jobs boot it instead of the raw image. Compose
  failure is non-fatal — suite jobs fall back to the raw image and the
  legacy runtime-install path.
- The runtime "Install shell tools for common suite" step is retained
  only as that fallback (compose_image=false input or repository
  variable E2E_COMPOSE_IMAGE=0); on the composed path it is replaced by
  a verification-only step.
- Summary/screenshot naming keeps using the raw input image so
  run-scoped tags don't leak into artifact names.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KSjBPVDcnK71WveusaK7Nc
@hanthor

hanthor commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

Holding this one deliberately while merging the other four testsuite PRs in this batch.

ghost-lab is red here, but it's red on every open PR in this repo — #740, #741, #745, #746, #747, #748, #749 — including pure-docs changes that touch nothing the lab exercises. The check reports from a self-hosted Argo instance at 192.168.1.102:2746, which is unreachable outside the lab network. It's broken infrastructure, not a signal about any of these diffs. It's also not a required check: the main — merge queue ruleset requires only Lint & syntax, Behave dry-run, and pytest, all three of which pass here.

So I've queued #746, #747, #748 and #749, which are docs and test-only changes.

This PR is different and I'd rather not merge it on a red lab. "compose test software as a container layer instead of runtime install" changes how the e2e suite builds its test images — which is precisely what ghost-lab validates end to end. The three required checks are static (lint, a behave dry-run, pytest); none of them actually boots the composed image. Merging this specific change while the only check that would catch a regression is down means shipping it unverified.

Two ways forward, whichever is easier:

  1. Bring the lab back and let ghost-lab run this PR for real.
  2. If the lab is going to be down for a while, someone runs the composed-layer path manually once and posts the result here, and we merge on that evidence.

Worth fixing the reporting either way: a check that has been failing on 100% of PRs for days is indistinguishable from a check that's finding real bugs, and right now it's training everyone to ignore it. If the lab is expected to be offline, the reporter should skip or report neutral rather than fail.

@castrojo castrojo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by @castrojo for Hive auto-merge on green CI.

@castrojo
castrojo added this pull request to the merge queue Aug 25, 2026
Merged via the queue into main with commit ffc7d81 Aug 25, 2026
6 of 7 checks passed
pull Bot pushed a commit to joshyorko/testsuite that referenced this pull request Aug 27, 2026
… var (projectbluefin#754)

The installer suite's @luks scenario asserts that the projectbluefin/common#385
`rd.luks.name=` parsing fix holds on a real installed system. It gated on a
`LUKS_ENABLED` environment variable — and nothing in the repo has ever set it.

The suite runs through the `common|lifecycle|installer` branch of e2e.yml, which
exports a fixed list (VM_IP, VM_USER, SSH_KEY, SSH_PORT, ZSTD_CHUNKED) and the
reusable workflow exposes no input to add another, so a downstream repo cannot
supply it either. `git grep LUKS_ENABLED` finds two lines, both readers, no
writer. The scenario has therefore skipped on every run since it landed, and a
skipped scenario reads as green: the assertion was never once exercised, while
projectbluefin/dakota#651 tracked it as implemented.

Replace the dead gate with a runtime probe. `target_uses_luks()` runs
`lsblk -rno TYPE | grep -qx crypt` on the DUT — a booted LUKS system always has
an unlocked dm-crypt mapping — and caches the answer on the context, so one SSH
round trip covers the run. The scenario now runs exactly when the target is a
LUKS install and still skips cleanly everywhere else, including on today's
direct-kernel-boot QEMU lane where nothing changes.

`LUKS_ENABLED` remains an explicit override in both directions, so an operator
can force the scenario on to prove a regression or off to keep it out of a run.
The behaviour change is what an unset variable means: "probe the target" rather
than "no LUKS".

No workflow change, deliberately: .github/workflows/e2e.yml is owned by open
PR projectbluefin#744, and a probe is the better fix regardless — it cannot silently disagree
with the machine it is describing.

tests/unit/test_installer_environment.py pins the probe, the override, the
caching, and a guard that fails if the gate reverts to reading an unset variable
alone. 21 of its 23 tests fail against the previous implementation.

Learning written back to docs/skills/ci-ops/e2e-workflow/ as a new
references/installer-suite.md, with the general rule: an env-var gate is only as
real as the plumbing that sets it — grep for the writer, or probe the DUT.

Refs: projectbluefin/dakota#651, projectbluefin/common#385

Assisted-by: Claude Opus 5 via Claude Code

Co-authored-by: Danathar <Danathar@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants