test(smoke): skip @vm_only scenarios in container lanes - #740
Conversation
offline_boot.feature drives the device under test entirely over SSH -- its own header says "Runner: plain SSH behave". Container lanes run behave *inside* the nested target and have no sshd, so every scenario failed its Background with Cannot reach VM at 127.0.0.1 over SSH after 5 attempts: rc=255 Those failures were invisible while the GNOME readiness gate kept container lanes from executing any scenario at all; they report a runner limitation as an image regression. Tag the feature @vm_only and skip it in the smoke hook when the lane is a container lane. Detection uses /run/.containerenv, which podman writes into every container it starts and a KubeVirt VM never has. The existing _IN_CONTAINER heuristic cannot be reused here: it probes for /usr/bin/bootc, which is present in the container lane too because the target *is* a bootc image. No timeout is lengthened and no assertion is weakened -- the scenarios still run unchanged in VM lanes, where they are meaningful. Assisted-by: Claude Opus 5 via GitHub Copilot CLI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
hanthor
left a comment
There was a problem hiding this comment.
Straightforward and correctly scoped. offline_boot.feature is genuinely SSH-only per its own header, and a container lane has no sshd to reach — the Cannot reach VM ... over SSH failure really is a runner-capability mismatch being misreported as an image regression, not a weakened assertion.
The new /run/.containerenv detection in tests/shared/runtime_env.py is a better signal than the existing _IN_CONTAINER heuristic (which probes for /usr/bin/bootc — present in both lanes since the container-lane target is itself a bootc image); the PR includes a live-lane confirmation of that distinction in the description, and it's independently plausible since podman writes that marker into every container it starts.
No timeouts are lengthened, no assertions are dropped, and the tag only skips scenarios that cannot be meaningful in a container lane — this is a coverage-reporting fix, not a coverage reduction: the scenarios still run fully in VM lanes. TestInContainerLane unit tests cover the marker-present/absent/path cases directly. GitHub Actions checks (lint, behave dry-run, pytest, coverage-snapshot freshness) are all green on this one.
The ghost-lab external status is red here too, but it's red identically across every other open PR in this review batch (including this one, which has otherwise clean CI and a trivial, well-reasoned diff), so I'm reading that as broken/flaky external lab infra rather than a signal about this change.
Generated by Claude Code
offline_boot.featuredrives the device under test entirely over SSH — its ownheader says "Runner: plain SSH behave (no qecore — no GUI interaction needed)".
Container lanes run behave inside the nested target and have no sshd, so all 8
scenarios failed their Background with:
These were invisible while the GNOME Shell readiness gate kept container lanes from
executing any scenario at all (fixed in projectbluefin/lab#614/#615). They report a
runner limitation as an image regression.
Change
offline_boot.feature@vm_only.@vm_onlyscenarios in the smokebefore_scenariohook when the lane is acontainer lane, following the existing
@bluefin/@dakota_only/@wifiskippattern already in that hook.
tests/shared/runtime_env.pydetects a container lane via/run/.containerenv,which podman writes into every container it starts and a KubeVirt VM never has.
The existing
_IN_CONTAINERheuristic cannot be reused: it probes for/usr/bin/bootc, which is present in the container lane too because the target isa bootc image. Confirmed on a live lane:
Not done
No timeout is lengthened and no assertion is weakened or deleted. The scenarios still
run unchanged in VM lanes, where they are meaningful. Scenario totals are unchanged
(skipping is not removal), so the coverage snapshot is already current.
Companion infrastructure fixes (
/dev/uinputmode,setuptoolsfor qecore'spkg_resourcesimport) are in projectbluefin/lab#619 — deliberately kept in aseparate PR per the repository-ownership boundary.
Validation
ruff check tests/ --select E,F,W --ignore E501✅behave --dry-run tests/smoke/features✅ (25 features, 196 scenarios)python3 -m pytest tests/unit/ -q→ 1255 passed ✅scripts/update_coverage_snapshot.py→ "No change — snapshot already current"Assisted-by: Claude Opus 5 via GitHub Copilot CLI