Skip to content

tests: 7 unit tests fail on clean main (zot policy, build metrics, page datasets, recc seam) #635

Description

@castrojo

What

pytest tests/unit/ on pristine origin/main (d6243bb, 2026-08-09) fails 7 tests; 357 pass. The Test Suite Validation workflow is intermittently red on main pushes, and because these failures are ambient, real regressions get lost in the noise.

Failing tests

  • test_zot_cache_policy.py::test_zot_sync_is_on_demand_and_bounded — asserts no "**" sync prefix, but zot-cache.yaml uses "**" for fedora/k8s/ecr/lscr registries. Test and manifest disagree on the bounded-prefix policy.
  • test_build_metrics.py::test_zot_metrics_are_enabled_on_both_registries — expects annotation sync-policy-v2, manifest has sync-policy-v4.
  • test_page_dataset_collector.py::test_tests_matrix_derives_rows_from_surface_and_resultsrows_with_completed_runs expected 14, got 15 (data drift vs. hardcoded counts).
  • test_page_dataset_collector.py::test_applications_matrix_keeps_bazaar_fallbacks_explicit — same class of data drift.
  • test_recc_runner_seam.py ×3 — recc overlay/runner seam assertions no longer match the manifests.

Reproduction

git worktree add /tmp/lab-main-check origin/main --detach
cd /tmp/lab-main-check && python3 -m pytest tests/unit/ -q

Fix

For each test, decide whether the test or the manifest is the current truth (e.g. was widening to ** prefixes deliberate? was the sync-policy-v4 bump deliberate?) and update the losing side. The page-dataset tests likely need counts derived from fixtures instead of hardcoded.

Related follow-up (from the 2026-08 bandwidth session)

  • Re-measure uplink with scripts/traffic_report.py --window 24h a day after PR chore(lab): cut uplink bandwidth — upstream digest polls, suspend unreviewed crons #632 merged; baseline was ~170–225 GiB/day and should collapse.
  • The ~1 TiB spike days (2 of last 5) were never definitively attributed — prime suspect is cold BuildStream/BST source fetches. Verify with du -s on the buildbarn CAS after the next spike; zot exports pull counts, not bytes.

Automatable: yes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions