Skip to content

fix(sign): trust cosign's unsigned answer only when the registry lists no signature bundle - #1239

Merged
teemow merged 1 commit into
mainfrom
sign-precheck-1237
Sep 19, 2026
Merged

teemow merged 1 commit into
mainfrom
sign-precheck-1237

Conversation

@teemow

@teemow teemow commented Sep 19, 2026

Copy link
Copy Markdown
Member

Closes #1237.

What

retagger sign verifies a digest before signing it and signs when the verify fails. The pre-check now

  • logs the verify's error at debug level, with the reference and cosign's message;
  • trusts cosign's "no signatures found" only once the registry confirms it lists no signature bundle among the digest's referrers (hasSignatureBundle, a filtered referrers listing with the credentials cosign uses). A listed bundle means cosign's lookup was throttled: the verify is retried with the usual backoff, up to cosignAttempts times;
  • signs over another identity's signature without asking the registry (cosign's answer is faithful there), and fails the tag for any other verify error instead of signing blindly;
  • counts the race between concurrent cosign processes refreshing the shared TUF cache (failed to persist metadata) as transient.

The retry loop of cosign() moved into retrying(), shared with the referrers listing. go-containerregistry is the new dependency for the listing.

Why

cosign v3 verify auto-detects the bundle format with one referrers listing and, when that listing fails for any reason, silently switches to the legacy signature-tag path, which reports the digest unsigned. The registry throttles referrers listings per identity (1000 per 60 s, TOOMANYREQUESTS); a sign-all pass with four workers trips that limit about every 80 s, and the four in-flight pre-checks then read "no signatures found" and sign the digests again. Reproduced with 36 concurrent verifies of signed digests: 235 of 261 failures were "no signatures found", 17 the explicit TOOMANYREQUESTS, the rest the TUF cache race.

Verification

  • Unit tests: the pre-check against an in-memory registry with the referrers API (unsigned → signed, one listing; bundle listed → verify retried, no sign; another identity → signed, no listing; unrelated verify error → failed, one cosign call); the listing under throttling (retried); the artifact-type filter.
  • After the merge: one sign-all pass; the skopeo-file jobs must report 0 signed for tags whose digest did not change. Result on sign: a failing pre-check verify is silent and re-signs the digest #1237.

…s no signature bundle

cosign verify detects the bundle format with a referrers listing and, when
that listing fails for any reason -- the registry throttling the per-identity
referrers lookups of a busy pass -- falls back silently to the legacy signature
tag and reports "no signatures found". The pre-check took that for an unsigned
digest and signed it again: a duplicate signature and Rekor entry per affected
digest, in batches of one per worker every time the throttle tripped.

The pre-check now logs the verify's error at debug level, and confirms an
"unsigned" answer against the registry's referrers list before signing:
a listed signature bundle means the lookup was throttled and the verify is
retried with backoff; a listing that fails for good, or a verify that fails
for a reason other than an unsigned digest or another identity's signature,
fails the tag instead of signing it. The TUF cache race between concurrent
cosign processes ("failed to persist metadata") joins the transient set.

Closes #1237
@teemow
teemow requested a review from a team as a code owner September 19, 2026 04:27
@teemow
teemow merged commit e3e34fd into main Sep 19, 2026
5 checks passed
@teemow
teemow deleted the sign-precheck-1237 branch September 19, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sign: a failing pre-check verify is silent and re-signs the digest

1 participant