feat(ui): the recipe preview shows one sample and the last step at full strength #1116

Workflow file for this run

	name: CI

	on:
	push:
	branches: [main]
	pull_request:
	# #49's benchmark, by hand. It is a wall-clock measurement, so it does not
	# belong on every pull request: a shared runner's numbers move for reasons
	# nobody chose, and gating on them would fail builds that are fine. The
	# hardware-independent half of the same question — how much work each gesture
	# asks the browser to do — is asserted in `e2e/perf.spec.ts` and does run on
	# every pull request, inside the job below it.
	workflow_dispatch:

	# Every job here reads the repository and writes nothing back to it: the two drift
	# gates run `git diff` locally, the artifacts go to the run rather than to a ref,
	# and no job touches an issue, a comment, a label or a release. So the token they
	# get is `contents: read` and every other scope is `none` — a `permissions:` block
	# sets what it does not name to `none`.
	#
	# Stated here rather than left to the repository setting it currently matches,
	# because that setting is a checkbox in another system: one click flips every
	# workflow in the repository back to a write-enabled token, and nothing in a diff
	# would record it. This is the same rule written where a reviewer reads it.
	#
	# It matters because of what this workflow deliberately does run: `uv sync` and
	# `pnpm install` execute build backends and lifecycle scripts from manifests the
	# pull request itself can edit, and the suites, the Playwright specs and the docker
	# build all run contributor code by design. That is correct for CI and it is why
	# the token in the same job must not be able to write anything.
	permissions:
	contents: read

	# A pull request that is pushed to three times in a minute currently starts three
	# full copies of this workflow — twelve jobs each, two of which download a browser
	# and one of which resolves torch — and the first two are answering questions about
	# a commit nobody is reviewing any more. Superseded runs for the same pull request
	# are cancelled; `main` is never cancelled, because a push there is the only run of
	# that commit that will ever exist.
	concurrency:
	group: ${{ github.workflow }}-${{ github.ref }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	jobs:
	python:
	runs-on: ubuntu-latest
	timeout-minutes: 30
	env:
	# ffmpeg is installed below. If that ever breaks, the video tests must go red — a
	# silently skipped video test looks exactly like a passing one.
	VISIONSET_REQUIRE_FFMPEG: "1"
	steps:
	# `persist-credentials: false` on every checkout in this file, and the reason
	# is the step that comes after it rather than this one. By default the action
	# leaves the job's token in `.git/config` as a credential for the remote, where
	# it outlives the checkout and is readable by everything else in the job — and
	# everything else in the job is, on purpose, code the pull request supplied: a
	# build backend, a lifecycle script, a test, a Playwright spec.
	#
	# Nothing here pushes. Both drift gates run `git diff` against the local
	# working tree, which needs no remote and no credential, so switching it off
	# costs this workflow nothing and removes the token from reach.
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	# The ubuntu-24.04 runner routes apt through /etc/apt/apt-mirrors.txt with
	# azure.archive.ubuntu.com first. When that mirror degrades it does not fail,
	# it stalls: apt pays a timeout per index before falling back, which is how a
	# fifteen-second step became a fifty-minute one and held a pull request behind
	# a check that was never going to arrive. Pin the fallback as the only mirror.
	# It is the canonical archive, and a mirrorlist whose first entry can silently
	# cost an hour is not a mirrorlist worth keeping.
	#
	# One file at a time, because the layout moves: this image carries `.sources`
	# and no `.list`, and it has been the other way round. Patching them in a single
	# `sed` reads as tidier and is the trap — an unmatched glob fails the whole
	# invocation, so one absent path leaves every file unpatched while the step
	# still reports success, which is the silent half-fix this exists to prevent.
	# The `[ -f ]` guard makes each file independent and makes an absent one a
	# no-op. The pattern matches the host alone, with no scheme, so it holds whether
	# the entry is written http or https.
	- name: Pin apt to archive.ubuntu.com
	run: \|
	for f in /etc/apt/apt-mirrors.txt /etc/apt/sources.list \
	/etc/apt/sources.list.d/.list /etc/apt/sources.list.d/.sources; do
	[ -f "$f" ] \|\| continue
	sudo sed -i 's\|azure\.archive\.ubuntu\.com\|archive.ubuntu.com\|g' "$f"
	done

	# `-o` options rather than bare `apt-get`, because the pin above removes the
	# mirror that stalls and these bound what any other mirror can cost. A
	# fifteen-second ceiling per request with three retries turns an unreachable
	# index into seconds of noise instead of an unbounded wait; without them apt's
	# own default is patient enough to outlast the job.
	#
	# ffmpeg is installed rather than assumed: unlike the browsers, it is not in
	# the runner image, and the video fixtures shell out to it.
	- name: Install ffmpeg
	run: \|
	sudo apt-get update -o Acquire::Retries=3 -o Acquire::http::Timeout=15
	sudo apt-get install -y --no-install-recommends \
	-o Acquire::Retries=3 -o Acquire::http::Timeout=15 ffmpeg

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	# `--locked`, not a bare `uv sync`, and it does two jobs. It refuses to
	# re-resolve, which is what keeps the dependency cool-down honest — a
	# resolution that happens here would be one nobody applied the rule to (see
	# scripts/cooldown.sh). And it fails when uv.lock has drifted from
	# pyproject.toml, so an uncommitted lock is a named red step rather than a
	# silent re-lock that makes CI test a set no reviewer chose.
	- name: Sync environment
	run: uv sync --locked

	- name: Ruff (lint + format)
	run: \|
	uv run ruff check .
	uv run ruff format --check .

	# One invocation: the kernel's strict flags are per-module config in
	# pyproject.toml, so `mypy src/visionset` already checks the kernel strictly.
	- name: Mypy (strict kernel, full src)
	run: uv run mypy src/visionset

	- name: Import contracts (import-linter)
	run: uv run lint-imports

	- name: MCP tool reference drift gate
	run: uv run python scripts/export_mcp_tools.py --check

	- name: Export target catalog drift gate
	run: uv run python scripts/export_target_catalog.py --check

	# `-n auto` resolves to the runner's core count. The suite has no expensive
	# test to remove — ~63 ms mean, no fat tail — so parallelism is the only
	# thing that makes this step faster, and it is ~90% of the run's critical
	# path when serial.
	- name: Pytest
	run: uv run pytest -n auto

	# The smoke test above already calls into the example; this proves it also
	# runs as a script from a clean checkout, images and all.
	- name: SDK end-to-end example
	run: uv run python examples/sdk_end_to_end.py

	# The ingest half of the same argument, and the one that needs the ffmpeg
	# installed above: this example generates its own clip.
	- name: Ingest end-to-end example
	run: uv run python examples/ingest_end_to_end.py

	# M3's exit criterion: the same cycle — ingest, annotate, release — driven three
	# ways from outside the process. One job per surface rather than three steps in
	# one, so a red X names the surface that broke; `fail-fast: false` is what keeps
	# one broken leg from cancelling the other two and reporting on a third of the
	# criterion. Stills only in all three, so none of them needs ffmpeg.
	#
	# Each Python leg also runs inside `uv run pytest` above, as a smoke test
	# asserting on the frozen Summary it returns. This is the other half of that
	# pair: the only thing that proves the installed console script works from a
	# clean checkout, which an in-process runner cannot.
	e2e:
	name: e2e (${{ matrix.surface }})
	runs-on: ubuntu-latest
	timeout-minutes: 15
	strategy:
	fail-fast: false
	matrix:
	include:
	- surface: http
	run: uv run python examples/http_end_to_end.py
	- surface: cli
	run: uv run bash examples/cli_end_to_end.sh
	- surface: mcp
	run: uv run python examples/mcp_end_to_end.py
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Sync environment
	run: uv sync --locked

	# `uv run` puts the virtualenv's bin/ on PATH, so `visionset` is the
	# installation under test — which the HTTP leg starts a server with and the
	# MCP leg spawns a server with.
	- name: Drive the cycle over ${{ matrix.surface }}
	run: ${{ matrix.run }}

	frontend:
	runs-on: ubuntu-latest
	timeout-minutes: 20
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm

	- name: Install
	run: pnpm install --frozen-lockfile

	- name: Version sync gate
	run: pnpm version:check

	# The spec this reads is kept fresh by the `openapi` job and by the pytest gate in the
	# `python` job, so this one needs no Python. Two independent links: spec matches the app,
	# client matches the spec.
	- name: Generated client drift gate
	run: \|
	pnpm generate:client
	git diff --exit-code frontend/ui-core/src/generated \|\| {
	echo "::error::The generated API client is stale — run 'pnpm generate:client' and commit the result."
	exit 1
	}

	# Build before lint, and the order is load-bearing. `frontend/app` resolves
	# `@visionset/annotator` through its `dist/` — the workflow gotcha #47
	# recorded — so on a clean checkout the engine has no type declarations until
	# something builds it. That first bit when #49 put `bench/` into
	# `tsconfig.e2e.json`: the benchmark imports the scene, the scene imports the
	# engine's `Annotation` type rather than hand-writing a copy of a kernel
	# shape, and `pnpm run typecheck:e2e` answered TS2307 for a module that is
	# right there in the workspace.
	- name: Build
	run: pnpm -r build

	- name: Lint
	run: pnpm -r lint

	- name: Test
	run: pnpm test

	# M4's behavioural contract: v1's Playwright specs, re-pointed at the annotator
	# demo. A job of its own rather than a step in `frontend`, because root
	# `pnpm test` is `pnpm -r test` and hanging a browser off that would put a
	# ~100 MB download in front of every unit-test run. The `e2e` matrix above is the
	# precedent — a job that pays for its own setup — and the name says `annotator`
	# so a red X in the checks list names the surface rather than colliding with it.
	annotator-e2e:
	name: annotator e2e (chromium)
	runs-on: ubuntu-latest
	timeout-minutes: 20
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm

	- name: Install
	run: pnpm install --frozen-lockfile

	# Keyed on the version the lockfile actually resolved, never on the range in
	# package.json: a caret bump that changed the browser build while the key
	# stayed still would restore a cache for the wrong revision, and Playwright
	# refuses to run against one.
	- name: Resolve the Playwright version
	id: playwright
	run: \|
	version=$(node -p "require('./frontend/app/node_modules/@playwright/test/package.json').version")
	echo "version=$version" >> "$GITHUB_OUTPUT"

	- name: Cache browsers
	id: browsers
	uses: actions/cache@v6
	with:
	path: ~/.cache/ms-playwright
	key: ms-playwright-${{ runner.os }}-${{ steps.playwright.outputs.version }}

	# No `--with-deps`, deliberately. That flag is an `apt-get` hidden behind a
	# browser installer, and apt is the entire reason this step has hung: the
	# runner's mirrorlist puts a mirror first that stalls rather than fails, and
	# one afternoon it turned a fifteen-second step into a fifty-minute one.
	#
	# The libraries it would install are already here. This image ships Google
	# Chrome, Chromium, Edge and Firefox, and none of those could be installed
	# without the shared libraries a Chromium build links — so `--with-deps` asks
	# Ubuntu's archive for packages the machine already has. What Playwright does
	# still need is its own pinned build, which comes from Playwright's CDN rather
	# than from apt, and which is exactly what the cache above holds.
	#
	# If a library ever is genuinely absent, the browser fails to launch in the run
	# below and says so. That is a loud failure at a named step, which is the kind
	# worth having; the alternative was a silent stall with no output at all.
	- name: Install chromium
	if: steps.browsers.outputs.cache-hit != 'true'
	run: pnpm --filter @visionset/app exec playwright install chromium

	# `frontend/app` resolves `@visionset/annotator` and `@visionset/ui-core`
	# through their `dist/`, so an unbuilt workspace package is served silently
	# rather than failing to compile — and on a clean checkout that is not one
	# failing scenario, it is all of them hunting for a blank page. The Playwright
	# config builds both too; doing it here as well keeps the failure at a named
	# step instead of inside a web-server log. Dependency order: `ui-core`
	# re-exports `classColor` from the annotator.
	- name: Build the workspace packages
	run: \|
	pnpm --filter @visionset/annotator build
	pnpm --filter @visionset/ui-core build

	# Four workers to match the runner's four cores; the config's own CI
	# fallback is two, sized for a smaller runner generation.
	- name: Drive the demo
	run: pnpm --filter @visionset/app e2e
	env:
	VISIONSET_PW_WORKERS: "4"

	- uses: actions/upload-artifact@v7
	if: failure()
	with:
	name: playwright-report
	path: frontend/app/playwright-report/
	retention-days: 7

	# #59: the whole cycle in a browser, against a real server and a real kernel.
	#
	# A job of its own, and it needs both toolchains — this is the only check in the
	# repository that does: `uv` for the server under test, `pnpm` for the bundle it
	# serves. Everything else is one or the other, which is why neither existing job
	# could have grown this leg.
	#
	# It drives the built artifact at `/app/`, exactly as the wheel serves it, so
	# it is also the only check that exercises the static mount and #58's SPA
	# deep-link fallback for real.
	cycle-e2e:
	name: browser cycle (chromium)
	runs-on: ubuntu-latest
	timeout-minutes: 30
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Sync environment
	run: uv sync --locked

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm

	- name: Install
	run: pnpm install --frozen-lockfile

	# Keyed on the resolved version, never the range — the same reasoning as the
	# annotator's job, and the same cache.
	- name: Resolve the Playwright version
	id: playwright
	run: \|
	version=$(node -p "require('./frontend/app/node_modules/@playwright/test/package.json').version")
	echo "version=$version" >> "$GITHUB_OUTPUT"

	- name: Cache browsers
	id: browsers
	uses: actions/cache@v6
	with:
	path: ~/.cache/ms-playwright
	key: ms-playwright-${{ runner.os }}-${{ steps.playwright.outputs.version }}

	# No `--with-deps`, deliberately — the annotator e2e job's install step
	# carries the full reasoning; it applies unchanged here.
	- name: Install chromium
	if: steps.browsers.outputs.cache-hit != 'true'
	run: pnpm --filter @visionset/app exec playwright install chromium

	# The config's own webServer builds the engine, the design system and the
	# bundle, copies it into the package data, creates a throwaway workspace and
	# starts `visionset server`. One command, so a failure in any of those steps is
	# reported at this step rather than inside a server log.
	- name: Drive the whole cycle
	run: pnpm --filter @visionset/app cycle

	- uses: actions/upload-artifact@v7
	if: failure()
	with:
	name: cycle-report
	path: frontend/app/playwright-report/
	retention-days: 7

	# #66: the delivery thesis, built and then checked as an artifact. pip is the
	# vehicle and the compiled UI travels inside the single wheel, and three things
	# can be wrong with that wheel which no source test can see — the bundle is not
	# in it, the version metadata disagrees with VERSION, or something enormous came
	# along. Each installs cleanly, starts cleanly, and is wrong.
	#
	# The wheel and sdist are uploaded from every build, so a main build always
	# leaves something installable behind. VISIONSET_REQUIRE_WHEEL turns the local
	# skip into an error here, the VISIONSET_REQUIRE_FFMPEG rule.
	wheel:
	name: wheel (build, install, serve)
	runs-on: ubuntu-latest
	timeout-minutes: 20
	env:
	VISIONSET_REQUIRE_WHEEL: "1"
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Install
	run: pnpm install --frozen-lockfile

	# The whole pipeline in one script rather than a chain here, because the
	# ordering is load-bearing: `uv build` copies `_static/` at the moment it
	# runs, so a build before `bundle:static` ships no app and says nothing.
	- name: Build the wheel and sdist
	run: bash scripts/build_dist.sh

	- name: Install it in a clean venv and serve the app
	run: uv run pytest tests/packaging -q

	- name: Upload the distribution
	uses: actions/upload-artifact@v7
	with:
	name: visionset-dist
	path: dist/*
	if-no-files-found: error

	# #67: the vision document's success metric, automated — install → project →
	# video → 50 boxes → release → YOLO export → a trainer loads it. The gate the
	# beta cannot ship without, and the one job here that runs against **what a
	# user gets from pip** rather than against the source tree: the wheel goes into
	# an empty virtual environment and nothing from the checkout is importable.
	#
	# It therefore proves in one run what a dozen source tests each prove a piece
	# of — the wheel, the entry points, plugin discovery, the media toolchain, and
	# every service in the cycle.
	#
	# `examples/thirty_minute_flow.py` names and times each stage and asserts its
	# own wall clock, so a failure here says which step broke rather than leaving a
	# traceback into whichever service happened to raise.
	thirty-minute-flow:
	name: 30-minute flow (wheel, end to end)
	runs-on: ubuntu-latest
	timeout-minutes: 45
	env:
	# The clip is generated rather than committed, and the export is checked by
	# the tool it exists for. Both are errors here rather than skips: a gate
	# that quietly stopped checking half of itself is worse than no gate.
	VISIONSET_REQUIRE_ULTRALYTICS: "1"
	YOLO_OFFLINE: "true"
	MPLBACKEND: Agg
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	# The mirror pin, argued in the `python` job at the top of this file.
	- name: Pin apt to archive.ubuntu.com
	run: \|
	for f in /etc/apt/apt-mirrors.txt /etc/apt/sources.list \
	/etc/apt/sources.list.d/.list /etc/apt/sources.list.d/.sources; do
	[ -f "$f" ] \|\| continue
	sudo sed -i 's\|azure\.archive\.ubuntu\.com\|archive.ubuntu.com\|g' "$f"
	done

	# `-o` options rather than bare `apt-get`, because the pin above removes the
	# mirror that stalls and these bound what any other mirror can cost. A
	# fifteen-second ceiling per request with three retries turns an unreachable
	# index into seconds of noise instead of an unbounded wait; without them apt's
	# own default is patient enough to outlast the job.
	#
	# ffmpeg is installed rather than assumed: unlike the browsers, it is not in
	# the runner image, and the video fixtures shell out to it.
	- name: Install ffmpeg
	run: \|
	sudo apt-get update -o Acquire::Retries=3 -o Acquire::http::Timeout=15
	sudo apt-get install -y --no-install-recommends \
	-o Acquire::Retries=3 -o Acquire::http::Timeout=15 ffmpeg

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Install
	run: pnpm install --frozen-lockfile

	- name: Build the wheel
	run: bash scripts/build_dist.sh

	# A venv with the wheel and the trainer in it, and nothing else — in
	# particular not the repository, which is what makes this a test of the
	# artifact. `--no-cache` so a previously built wheel of the same version
	# cannot be served from uv's cache in place of this one.
	#
	# The one place in this workflow that genuinely resolves a third-party
	# package from an index — `ultralytics` is named with no version and comes
	# from PyPI, unlike everything else here, which comes from uv.lock. So it
	# is the one place the cool-down wrapper is needed rather than decorative.
	# The local wheel is a path, and a path has no upload date, so the cutoff
	# cannot touch the artifact under test.
	- name: Install the wheel into a clean environment
	run: \|
	uv venv /tmp/flow-venv
	bash scripts/cooldown.sh \
	uv pip install --no-cache --python /tmp/flow-venv dist/*.whl ultralytics

	# `cd /tmp` so an accidental relative import cannot reach the checkout: the
	# script is named by absolute path and runs with the repository not on
	# its working directory.
	- name: Run the thirty-minute flow
	run: \|
	cd /tmp
	/tmp/flow-venv/bin/python "$GITHUB_WORKSPACE/examples/thirty_minute_flow.py" /tmp/flow

	# The other half of the inference matrix, on the `format-smoke` mould below: a
	# job of its own because the `local-inference` extra is roughly two gigabytes of
	# CUDA wheels, which is a cost nobody in the `python` job is paying for.
	#
	# That job installs the base distribution deliberately and is untouched by
	# this one. It is the half that proves a machine without the runtime imports
	# none of it and refuses with the install command rather than an `ImportError`.
	# What it cannot do is exercise the code that only exists when the runtime is
	# present — and until this job, nothing did: `tests/architecture/
	# test_optional_runtime.py` asserts that importing the server, the CLI, the job
	# registry and `visionset.inference` leaves torch, torchvision, transformers,
	# accelerate and huggingface_hub out of `sys.modules`, which is true by
	# construction on a machine where none of them is installed. A module-level
	# `import torch` under `visionset/inference` — or a lock bump that broke the
	# adapter, family resolution or the download path — would have passed green.
	#
	# `uv sync --locked --extra local-inference`: from the lockfile, never from a
	# side index, for the reason the `python` job gives — a resolution happening
	# here would be one nobody applied the cool-down to. The locked torch wheels
	# carry CUDA and run perfectly well on a runner that has no device.
	#
	# It runs whole directories, unlike `format-smoke`, and the difference is the
	# shadowing that job's comment describes: none of these five distributions ships
	# a top-level `tests` package, so this repository's namespace one is intact and
	# `tests.fixtures` imports normally here.
	#
	# Its rendered name is a required status context in the `main` ruleset (#640).
	# That is a contract with a setting in another system: renaming this job, giving it
	# an `if:` or a `paths:` filter, or otherwise letting it stop reporting on some pull
	# requests wedges every merge on a check that will never arrive — green CI, and a
	# merge button that simply never becomes available. Change the ruleset in the same
	# pull request or do not change the name. CONTRIBUTING.md's release-gate section has
	# the procedure.
	#
	# It is required because it is the only job that installs the `local-inference`
	# extra: every other job runs against a base install where the runtime is absent by
	# design, so `tests/inference/` and the code paths that touch real tensors execute
	# here and nowhere else. #622 (a `.detach()` on a list of feature maps, every point
	# suggestion answering 500) and #611 (a mask crossing the port as bytes per row) both
	# sit on that path and were green everywhere else.
	inference-smoke:
	name: inference smoke (transformers, torch on cpu)
	runs-on: ubuntu-latest
	timeout-minutes: 45
	env:
	# A missing runtime is an error rather than a skip, the
	# VISIONSET_REQUIRE_FFMPEG rule: a job that exists to exercise the runtime
	# and quietly exercised nothing looks exactly like a passing one.
	#
	# It says nothing about a missing GPU. This runner has no CUDA device and
	# never will, so the one test that reproduces the half-precision finding on
	# real tensors asks for the runtime and the device as two separate
	# questions and skips here on the second — see `tests/inference/test_fp16.py`.
	VISIONSET_REQUIRE_LOCAL_INFERENCE: "1"
	# Every hub call in these tests is faked, and this is what keeps that true:
	# a test that regressed into a real fetch fails here instead of downloading
	# gigabytes or hanging on somebody else's uptime.
	HF_HUB_OFFLINE: "1"
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Sync environment with the local-inference extra
	run: uv sync --locked --extra local-inference

	# The inference surface, named directory by directory rather than left to
	# `testpaths`: the rest of the suite already runs in the `python` job, and
	# what is wanted here is the code that reads a config, resolves a family,
	# drives the hub client and converts tensors — with the real libraries
	# loaded rather than a stand-in.
	#
	# No `-q`: `addopts` in pyproject.toml already carries one, and a second
	# suppresses the count line this job exists to put on the record.
	- name: The inference surface, with the runtime present
	run: \|
	uv run pytest \
	tests/inference \
	tests/architecture/test_optional_runtime.py \
	tests/server/test_inference.py \
	tests/server/test_suggest.py \
	tests/cli/test_inference_commands.py \
	tests/jobs/test_weights_job.py \
	-rs

	# #62's and #63's second acceptance criteria: the tools the exporters exist to
	# feed actually load what they wrote. Its own job because `ultralytics` brings
	# torch — roughly two gigabytes — and putting that in front of every `uv sync`
	# for two tests is a cost nobody in the main job is paying for. The dependency
	# groups are `yolo` and `coco`, installed only here.
	#
	# VISIONSET_REQUIRE_ULTRALYTICS turns the local skip into an error, so a broken
	# install goes red rather than quietly shrinking the suite. Same rule, same
	# reason, as VISIONSET_REQUIRE_FFMPEG in the python job.
	#
	# It runs the smoke FILES, never the suite, and that is not thrift. The ultralytics
	# wheel ships a top-level `tests` package — `__init__.py`, `conftest.py` and
	# all — which installs into site-packages and shadows this repository's
	# namespace-package `tests/`. A regular package beats a namespace portion
	# wherever it is found, so no path ordering fixes it: in this environment
	# `import tests.fixtures` raises. Both smoke modules therefore import nothing
	# from `tests.`, and both docstrings say so.
	format-smoke:
	name: format smoke (ultralytics, pycocotools)
	runs-on: ubuntu-latest
	timeout-minutes: 20
	env:
	VISIONSET_REQUIRE_ULTRALYTICS: "1"
	VISIONSET_REQUIRE_PYCOCOTOOLS: "1"
	# Ultralytics writes a settings file and phones home on first use. Neither
	# is wanted in CI, and the second makes a green build depend on a third
	# party's uptime.
	YOLO_OFFLINE: "true"
	MPLBACKEND: Agg
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Sync environment with the format groups
	run: uv sync --locked --group yolo --group coco

	# No `-q`: `addopts` in pyproject.toml already carries one, and a second
	# suppresses the count line — which is the line that would show this job
	# quietly shrinking to nothing.
	- name: The reference readers load the exports
	run: uv run pytest tests/formats/test_ultralytics_roundtrip.py tests/formats/test_coco_smoke.py

	# #49's performance benchmark: frame times during pan, zoom and drag over 200
	# boxes and 20 polygons of 32 vertices on a 4K asset, plus a CPU-throttling
	# ladder that says how much headroom each gesture has.
	#
	# Manual only, which the issue explicitly blesses ("regression check kept cheap;
	# manual trigger is fine for the beta"). Two reasons it is not on the PR path:
	# the numbers are a property of the machine, and the run takes about a minute of
	# deliberately serialized work — one worker, no retries, because a benchmark
	# sharing a runner with itself is not measuring anything.
	#
	# The recorded baseline lives in `docs/content/annotations.md` and was taken on a
	# developer machine, which is what the acceptance criterion asks for. A run here
	# answers a different question — "did this change make it worse on the same
	# hardware as last time" — so compare a dispatch against a dispatch, never
	# against the documented laptop numbers.
	annotator-bench:
	name: annotator bench (chromium, manual)
	if: github.event_name == 'workflow_dispatch'
	runs-on: ubuntu-latest
	timeout-minutes: 30
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm

	- name: Install
	run: pnpm install --frozen-lockfile

	- name: Resolve the Playwright version
	id: playwright
	run: \|
	version=$(node -p "require('./frontend/app/node_modules/@playwright/test/package.json').version")
	echo "version=$version" >> "$GITHUB_OUTPUT"

	- name: Cache browsers
	id: browsers
	uses: actions/cache@v6
	with:
	path: ~/.cache/ms-playwright
	key: ms-playwright-${{ runner.os }}-${{ steps.playwright.outputs.version }}

	# No `--with-deps`, deliberately — the annotator e2e job's install step
	# carries the full reasoning; it applies unchanged here.
	- name: Install chromium
	if: steps.browsers.outputs.cache-hit != 'true'
	run: pnpm --filter @visionset/app exec playwright install chromium

	# No separate build step, unlike `annotator-e2e`: this suite's web server
	# builds the engine and the application, because it serves `vite preview`
	# and a production build is part of what is being measured.
	- name: Measure
	run: pnpm --filter @visionset/app bench

	- uses: actions/upload-artifact@v7
	if: always()
	with:
	name: bench-results
	path: frontend/app/bench-results.json
	retention-days: 30

	openapi:
	name: OpenAPI contract drift gate
	runs-on: ubuntu-latest
	timeout-minutes: 15
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- name: Install uv
	uses: astral-sh/setup-uv@v7
	with:
	enable-cache: true

	- name: Sync environment
	run: uv sync --locked

	- name: Regenerate openapi.json and fail on drift
	run: \|
	uv run python scripts/export_openapi.py
	git diff --exit-code openapi.json \|\| {
	echo "::error::openapi.json is stale — run 'uv run python scripts/export_openapi.py' and commit the result."
	exit 1
	}

	# The documentation site. It builds `docs/content/` through Astro and Starlight, which is
	# the one thing no other job here does — every other suite reads the documentation
	# as text, and `tests/scripts/docs_links.test.mjs` checks its links before the
	# site rewrites them.
	#
	# Three assertions, in an order that matters, and `scripts/check.sh docs` runs the
	# same three:
	#
	# build every page compiles — a document that breaks Starlight is a
	# document nobody can read, and it is invisible on GitHub
	# determinism the projection the build produced is what a fresh projection
	# produces. Never before the build: `docs/src/content/
	# docs/` is generated and git-ignored, so on a clean checkout
	# there is nothing to be current with
	# internal links every link in `dist/`, anchors included — the half the
	# Markdown gate structurally cannot see
	#
	# No Python. It reads `docs/content/`, which is committed, and installs only the
	# documentation site's own dependencies — a separate pnpm workspace root, so
	# nothing here can move the application's lockfile. See docs/README.md.
	docs-site:
	name: docs site
	runs-on: ubuntu-latest
	timeout-minutes: 15
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- uses: pnpm/action-setup@v6 # reads packageManager from package.json

	- uses: actions/setup-node@v7
	with:
	node-version-file: .nvmrc
	cache: pnpm
	# The documentation site keeps its own lockfile, so the default —
	# the root's — would key the cache on a file this job never installs from.
	cache-dependency-path: docs/pnpm-lock.yaml

	# `--frozen-lockfile` for the reason every install in this file uses it: a
	# build that silently resolves something the lockfile does not name is one
	# nobody reviewed, and it is what the three-day cool-down exists to prevent.
	- name: Install
	run: pnpm --dir docs install --frozen-lockfile

	# The `docsSource()` integration projects `content/` into the content collection
	# as part of this command, so there is no separate sync step to forget and no
	# way for the build to be reading a stale copy.
	- name: Build
	run: pnpm --dir docs build

	- name: Projection is deterministic
	run: pnpm --dir docs sync:check

	- name: Internal links resolve
	run: node docs/scripts/check-links.mjs

	# The only job that touches docker/. Every other job installs the environment its
	# own way, so until this existed the dev stack was unverified by construction —
	# which is how its image came to be missing ffmpeg without a single red build.
	#
	# The image, not the compose file: `up` would need the frontend image, nginx and
	# a browser to say anything, while what can actually rot here is what the api
	# image contains.
	docker:
	name: dev image (docker/api.Dockerfile)
	runs-on: ubuntu-latest
	timeout-minutes: 15
	steps:
	- uses: actions/checkout@v7
	with:
	persist-credentials: false

	- uses: docker/setup-buildx-action@v4

	- name: Build the api dev image
	uses: docker/build-push-action@v7
	with:
	context: .
	file: docker/api.Dockerfile
	load: true
	tags: visionset-api-dev:ci
	cache-from: type=gha
	cache-to: type=gha,mode=max

	- name: Both binaries are on PATH
	run: \|
	docker run --rm visionset-api-dev:ci ffprobe -version
	docker run --rm visionset-api-dev:ci ffmpeg -version

	# The assertion that means something. A version banner proves a package is
	# installed; this proves the kernel can reach it — `_require_tool`, the probe,
	# the frame extraction, PYTHONPATH and the dev-group venv, all of it. Run the
	# way docker/compose.yaml runs the container, source bind-mounted at
	# /workspace, so a change to either file is exercised the way it will be used.
	#
	# VISIONSET_REQUIRE_FFMPEG=1 for the reason the `python` job sets it: without
	# it a missing binary skips the suite, and a silently skipped video test looks
	# exactly like a passing one — which is precisely the failure being guarded.
	#
	# `--user` on both bind-mounting steps, and it is the same argument the compose
	# file makes: a container writing into a mounted tree as root leaves files
	# behind that their owner cannot remove. A runner is thrown away, so what is at
	# stake here is not cleanup but agreement — this is the one place the image is
	# exercised the way the stack runs it, and an image that only works as root
	# would pass a gate that never tried anything else. A runner checks out as its
	# own uid rather than 1000, so the flag also proves the image does not depend
	# on the id it was built for. HOME with it, because the home directory the
	# image creates belongs to that other id.
	- name: The image can decode video
	run: \|
	docker run --rm \
	--user "$(id -u):$(id -g)" \
	-e HOME=/tmp \
	-v "$PWD:/workspace" \
	-e VISIONSET_REQUIRE_FFMPEG=1 \
	visionset-api-dev:ci \
	pytest tests/kernel/test_video_processor.py -q

	# The same shape of gap as the ffmpeg one above, and it had the same cause:
	# every other job installs visionset, so every entry-point assertion in the
	# suite was true by construction everywhere it was ever run, and none of them
	# ran here. The image shipped reaching the source by PYTHONPATH and installing
	# no distribution metadata — which cost the dev stack every one of its export
	# formats and stamped "0.0.0" into published releases, without a red build.
	#
	# Bind-mounted like the step above so it exercises the image the way
	# docker/compose.yaml runs it. `tests/formats/test_entry_points.py` is here
	# deliberately as well as in the `python` job: it was already the discovery
	# test, and where it ran was the whole of what it was missing.
	- name: The image can see its own distribution metadata
	run: \|
	docker run --rm \
	--user "$(id -u):$(id -g)" \
	-e HOME=/tmp \
	-v "$PWD:/workspace" \
	visionset-api-dev:ci \
	pytest tests/packaging/test_installed_metadata.py \
	tests/formats/test_entry_points.py -q

	# The console script comes from `[project.scripts]` now that the project is
	# installed, rather than from a hand-written shim. `docker/api-dev.sh` calls
	# `visionset init` and `visionset token create` on first boot, so an image
	# where this is missing is one that cannot start a fresh workspace.
	- name: The console script is on PATH
	run: docker run --rm visionset-api-dev:ci visionset --help

Navigation Menu

Uh oh!

feat(ui): the recipe preview shows one sample and the last step at full strength #1116

Workflow file

feat(ui): the recipe preview shows one sample and the last step at full strength #1116

Uh oh!

Workflow file for this run