fix(security): fail closed when the snapshot checksum is unavailable

[barbatos2011]

Whether a snapshot was verified at all was decided by an attacker-controllable response. Found by an audit of develop at 0c654cd.

The problem

Download gated verification on pre.HasMD5Sidecar, a flag set from a plaintext HEAD of the .md5sum URL. Anyone on the path to a mainnet mirror — all bare-IP http:// — could answer that HEAD with a 404, and the tarball was then extracted with expectedMD5 empty, no error, exit 0. The CLI reported (md5 sidecar absent — not verified) as a successful download. The attacker's own response was, in effect, a switch that turned integrity checking off.

What changed

The sidecar is fetched unconditionally whenever the operator has not opted out. A 404, a non-200, or a transport failure returns a typed VerificationUnavailableError before any tarball byte is requested, so nothing is written.

Driven A/B against a binary built from develop:

sidecar behaviour	develop	this branch
valid digest	extracted, verified	identical
404	extracted, exit 0, attacker state on disk	refused, 0 files in destination
500	extracted, exit 0	refused, 0 files
transport reset	extracted, exit 0	refused, cause wrapped, 0 files

A request recorder confirms the refusal precedes any tarball GET.

The MCP snapshot_download tool had no opt-out at all, so it always extracted unverified on a 404 — it gains a no_verify arg alongside the CLI's existing --no-verify, which survives the --detach re-exec.

Does this break real downloads?

No. All eight mirrors in SourceTable were re-probed live and every one returns 200 with a coreutils-format body, so none of the in-repo callers — the recipe, build-nile-fixture.sh, the CI job — needs the opt-out, and none is changed here.

The docs that said sidecars are sometimes absent are corrected rather than deleted: they now say what actually happens.

Behaviour changes worth reviewing

• A download whose sidecar is genuinely unavailable now exits 1 with VERIFICATION_UNAVAILABLE instead of succeeding. The mainnet mirrors index a backup directory as soon as it exists, so a run landing between tarball and sidecar publication would now fail — the error's first suggestion is to retry, for exactly that reason.
• One failure path changes error_code from DOWNLOAD_ERROR to VERIFICATION_UNAVAILABLE; exit code stays 1.
• Adds one additive optional field to snapshot-download.schema.json, so SchemaVersion goes 1.12.2 → 1.12.3.

Testing

go test ./... -race -count=1, go vet ./... and gofmt clean on the branch, plus the A/B matrix above and a live probe of all eight mirrors.

Ordering

This overlaps #204, which touches the same verification block and also bumps SchemaVersion to 1.12.3. Suggest merging #204 first, then rebasing this and bumping to 1.12.4 with make snapshot-schema-baseline. Both are independently correct against develop as it stands.

[barbatos2011]

Rebased on top of #204 to resolve the schema-version and download.go collision. This branch now contains #204's three commits as well, so its diff shows both until #204 merges — review the last commit (fix(security): fail closed when the snapshot checksum is unavailable) for the change specific to this PR.

The two fixes are integrated rather than merely stacked: the sidecar is fetched unconditionally (fail-closed) and the cleartext-transport warning still fires before it, DownloadResult carries both verification_skipped and the sha256/plaintext_transport fields, and both surfaces' JSON payloads report all of them. SchemaVersion is 1.12.5 here (develop took 1.12.3, #204 takes 1.12.4) with a regenerated baseline.

One test fixture changed as a consequence: #204's transport tests used a fake mirror that served no .md5sum, which was fine when a missing sidecar meant "skip verification" but is now correctly a hard error. The fixture serves a real sidecar, so those tests still exercise what they were written for. TestDownload_MissingSidecarFailsClosed and TestDownload_WarnsOnPlaintextMirror both pass together.