[architect] behave suite environment contract is unenforced convention: 14 hand-wired environment.py, quarantine gate copy-pasted, timing wired in only 10/14

[kubestellar-hive]

Architecture Finding

Type: anti-pattern / missing-interface
Affected area: tests/*/features/environment.py (14 files), tests/shared/quarantine.py, tests/shared/timing.py

Every behave suite ships its own environment.py implementing the full behave hook
lifecycle (before_all / before_scenario / after_scenario, plus after_step /
after_all in some). There are 14 of them and no two are alike — 26 lines (nvidia)
to 469 lines (smoke). There is no shared base, no mixin, and no contract test.

The cross-cutting concerns that every suite is supposed to honour are therefore
copy-pasted rather than inherited, and they have already drifted:

suite	skip_quarantine	record_start / record_end
bazzite	yes	no
common	yes	yes
developer	yes	yes
dx	yes	yes
flatcar	yes	no
hardware	yes	yes
installer	yes	yes
kde-smoke	yes	yes
lifecycle	yes	yes
nvidia	yes	no
security	yes	no
smoke	yes	yes
software	yes	yes
vanilla-gnome	yes	yes

Supporting evidence of the copy-paste rather than shared-abstraction shape:

• skip_quarantine is imported inside before_scenario in all 14 files — a
  function-local import repeated 14 times, not a module-level shared entry point.
• Wiring order has already diverged: tests/kde-smoke/features/environment.py
  initialises context.command_stdout / context.ssh_rc / context.scenario
  before the quarantine gate (lines 158-164), while every other suite gates first
  and initialises after. Nothing detects which order is correct.
• _first_value is defined three times — tests/common/features/environment.py:23,
  tests/kde-smoke/features/environment.py:35, and tests/shared/ssh_config.py:28.
• Only 2 of 14 environment modules have any unit coverage at all
  (tests/unit/test_installer_environment.py, tests/unit/test_kde_smoke_environment.py).

There are also two distinct, undeclared execution substrates behind the same hook
names — qecore.sandbox.TestSandbox (bazzite, developer, software, and others) vs
raw SSH via tests/shared/ssh_steps (common, flatcar, hardware, installer,
lifecycle, nvidia, security) — with no interface naming which contract a suite
implements.

Impact

• tests/shared/quarantine.py is well unit-tested in isolation
  (tests/unit/test_quarantine.py) but nothing asserts that a suite actually calls
  it. A new suite that omits the gate will run @quarantine, @future,
  @pending and @hardware_blocked scenarios for real. Those tags exist precisely
  because those scenarios are known-failing, so the omission does not fail loudly at
  authoring time — it fails later as a red E2E gate that looks like a product
  regression.
• The same holds for every future cross-cutting concern: adding one means editing 14
  files and hoping none is missed, which is exactly how the timing gap above opened.
• Four suites silently produce no scenario timing records at all, so any
  timing/SLA analysis is computed over a partial and undeclared subset.

Recommendation

1. Extract the shared preamble into a single entry point (e.g.
   tests/shared/suite_env.py exposing standard_before_scenario(context, scenario))
   and have each suite delegate to it, keeping only its genuinely suite-specific
   logic. Fold the duplicated _first_value into the existing tests/shared/ssh_config
   definition.
2. In the meantime, enforce the convention with a contract test that walks
   tests/*/features/environment.py and asserts each before_scenario calls
   skip_quarantine, so a new suite cannot silently drop the gate. A hold-gated PR
   implementing step 2 only (new file tests/unit/test_suite_environment_contract.py,
   no runtime change) accompanies this issue.
3. Decide whether timing is mandatory. If it is, wire record_start/record_end
   into bazzite, flatcar, nvidia and security and extend the contract test to cover
   it. If it is not, document it as opt-in — see the companion orphaned-timing issue.

---

Filed by architect agent (ACMM L5 — hold-gated mode)

🐝 Hive Agent: architect | Instance: hosted-projectbluefin-knuckle-gjvq | SHA: ee82d53

— hive: agent=architect backend=copilot model=claude-opus-5