SSH certificate authorization is asserted by Warpgate, not verified by Vault

[theredspoon]

Context

#26 has been open since April 2022 asking whether and how Warpgate should get SSH certificate support. There have been real prior implementation attempts, not just discussion:

• Add Ssh cert auth for targets #1847 (still open) has Warpgate generate and hold its own ephemeral CA and sign temporary client certs itself, no external secrets manager involved, which the linked design discussion notes doesn't solve trust-on-first-use for that self-hosted CA;
• Feat/OIDC ephemeral ssh keys ttl #2186 (closed, not merged) took a narrower OIDC-ephemeral-key approach scoped to the client↔Warpgate leg.
• Add SSH certificate authentication for targets, issued by Vault #2397 is the current, most actively developed attempt, delegating certificate issuance to an external Vault/OpenBao CA instead of Warpgate holding one itself.

Reviewing it surfaced a specific design question worth resolving as part of settling #26, not just a bug in that one PR: how does Vault know the signing request it's honoring actually came from the session it looks like it came from?

The gap

Under #2397's design, Vault can't distinguish one target/session from another: role, principals, and key_id are all values Warpgate's own code asserts in the signing request, not anything Vault independently verifies.

Under the model #2397 replaces, compromising Warpgate's stored credentials was bounded by whatever was actually stored for actually-configured targets. Under the new one, a compromised Warpgate can request a cert for any role its Vault token is allowed to sign for, fleet-wide, non-revocable, since role defaults to one shared value across every target unless each is individually configured otherwise. That's a regression in worst-case blast radius versus the status quo, not just a smaller-than-ideal improvement.

Confirmed against the current code: any target without an explicit role falls back to one shared default_role from VaultConfig (warpgate-protocol-ssh/src/client/mod.rs:960), and sign_ssh_key takes role as a plain string with only path-safety validation, no per-target allowlist tying a role to a specific target. An operator can narrow this back down by deliberately scoping Vault policy per role instead of relying on the shared default, but nothing here defaults to or enforces that.

A possible direction

One way to narrow this, offered for discussion rather than as a finished spec, and specific to HashiCorp Vault or OpenBao rather than a backend-agnostic proposal (#2397 only talks to Vault's own HTTP API today, and #1847's self-hosted-CA approach would need an entirely different design, there's no external system there that could independently verify anything): Vault's SSH secrets engine supports identity templating on the role fields that actually matter here, allowed_users_template, default_user_template, allowed_extensions_template. That's the mechanism that would let Vault render the principal it signs for from the caller's own verified entity metadata, instead of trusting whatever valid_principals Warpgate puts in the request body. Confirmed present in OpenBao's OSS build too, not Enterprise-gated.

Vault/OpenBao side, if this is the right general shape:

• One entity per operator (warpgate-<username>), provisioned out-of-band, never written by Warpgate. Warpgate's own policy gets no write access to identity/*.
• Group membership decides which SSH role a session can sign against, not target config, so wg-prod and wg-dev groups map to separate signing roles with separate policies.
• A token role scoped to allowed_entity_aliases="warpgate-*", orphan=true, token_num_uses=1, short TTL (~60s), so Warpgate can mint session-scoped child tokens but never into an existing privileged entity.
• The base policy Warpgate's parent token holds explicitly denies the caller from setting valid_principals, key_id, critical_options, ttl, or cert_type in the signing request at all, via denied_parameters.
• SSH role config: default_user_template=true, default_user="{{identity.entity.metadata.unix_user}}", allow_user_key_ids=false, allowed_critical_options="none" (fails open by default, so has to be set explicitly), 2-minute TTL.

Warpgate side:

• VaultClient::mint_session_token(warpgate_username, session_id), a new call that creates the scoped child token per session using the entity alias, never cached.
• sign_once's request body shrinks: valid_principals and key_id get deleted from what Warpgate sends, not renamed or left optional. Vault renders the principal itself from the entity.
• sign_ssh_key takes the session's actual authenticated username instead of formatting a free-text key_id string.

What that direction does not fix

Two limits on the above, both of which matter for how it gets described if it lands.

Scoped child tokens (allowed_entity_aliases, orphan=true, token_num_uses=1) only reduce blast radius if the compromised Warpgate process can't fall back to the broader parent credential it already holds. Remote code execution on that process gets the parent token, or the Kubernetes JWT / AppRole secret ID it would use to mint a new one, regardless of how narrowly any individual child token it requested was scoped. Child tokens alone don't close that path.

Identity templating on the SSH role doesn't close it either. allowed_entity_aliases="warpgate-*" is a wildcard: it bounds which pool of pre-provisioned operator entities a session's child token can bind to, not which one. Warpgate itself still supplies warpgate_username when minting that child token, so a compromised process can bind it to a different operator's entity than the one actually connected, the same self-assertion problem valid_principals has today, moved one level up.

So the direction above narrows blast radius, from any role fleet-wide to one of the provisioned operators, but doesn't by itself verify which operator a given session is. It changes where Warpgate makes an assertion about who a session belongs to. Warpgate is still the one making that assertion unilaterally, and nothing external verifies it, so it doesn't get implemented and treated as fully solving the problem.

Closing the gap: IdP-verified token auth to Vault

TL;DR: The gap: Vault trusts whatever role, principal, and key_id Warpgate asserts, with nothing to check it against. Verifying a real access token narrows that a lot — but doesn't close it, since Warpgate still decides which token maps to which request. Checked many plausible fixes; most fail for closely related reasons, since Warpgate constructs every request they'd rely on. The one thing that does close this specific piece: OpenBao's claim-templated role policy removes the field Warpgate would otherwise assert a role in — the Vault-compatible fallback (bound_claims) narrows this a lot but doesn't fully close it. Details below, collapsed by default.

Why this design uses the access token, not the ID token

Closing the gap means Vault stops trusting any Warpgate-originated claim about who's connecting, and instead verifies something the user's own identity provider signed. There's already a directly relevant, already-litigated discussion in this repo worth reading before designing this: #2300 (closed) proposed forwarding the SSO ID token to HTTP targets so a backend could exchange it with Vault/OpenBao's JWT auth for a per-user token, essentially this same idea applied to HTTP targets instead of SSH certificates. It was withdrawn by its own author after review, on the basis of real measurement against a live deployment behind a TLS-inspecting middlebox: forwarding the ID token required configuring the resource server's bound_audiences to accept Warpgate's own client ID, described in the PR itself as "the anti-pattern made load-bearing in config"; the token carried PII (email in sub) across the inspected boundary on every use; ID tokens carry no scope, so there's no way to bound what the forwarded credential authorizes; and RFC 8693 token exchange, tried as a mitigation, was blocked because the IdP required an access token as the exchange input, which Warpgate didn't have. OWASP ASVS 5.0 V10 is cited normatively: ID tokens shall only be consumed by the client that triggered the authorization flow. The replacement, #2309 (open), forwards the access token instead. RFC 8693 exchange with act set to Warpgate, proposed in #2300's own review thread as the correct long-term mechanism rather than in #2309 itself, was never implemented in either PR: #2300's IdP (WSO2) required an access token as subject_token, which Warpgate didn't have at the time; #2309 added that access token but never implemented the exchange step, just the retention and forwarding. #2309 carries an honest caveat of its own: switching token types alone doesn't fix PII exposure if the IdP puts it in the access token too, which it does in that deployment's case.

This design should follow that same correction rather than repeat the mistake #2300 already made and #2397's review almost repeated. What follows uses the access token as the baseline mechanism, not the ID token, checked against the current code.

What code has to change to carry the verified token from login to the certificate-signing call

Where this actually stands today. Warpgate's existing SSO/OIDC flow (crate warpgate-sso, used by warpgate-protocol-http) fully verifies and retains the ID token: SsoClient::finish_login (warpgate-sso/src/sso.rs:234-324) enforces signature, iss, exp, audience, and nonce, and the raw signed ID token is retained in SsoLoginState (warpgate-protocol-http/src/common.rs:49-54), persisted in the http_sessions table (plaintext, unencrypted at rest, warpgate-protocol-http/src/session.rs:103-133). The access token is not retained anywhere on main today; #2309 is the PR that adds that (SsoResult → SsoLoginResponse on the code-exchange path). This design depends on that landing, or equivalent access-token-retention work, rather than reinventing it.

There's also a directly reusable precedent for verifying a raw bearer JWT with no code exchange: SsoClient::verify_id_token / verify_id_token_to_response (warpgate-sso/src/sso.rs:330-392), currently only called by Kubernetes bearer auth. Useful groundwork, but it verifies an ID token specifically; the equivalent for a bearer access token would need to handle IdPs that issue opaque (non-JWT) access tokens, which auth/jwt-style verification can't consume directly. Worth checking early which shape the IdPs actually in use here issue, since that decides whether the baseline (bearer access token, verified as a JWT) is even reachable without the RFC 8693 exchange step.

SSH sessions can already require SSO approval via a browser bridge. A required SSO credential on a non-HTTP protocol gets rewritten to WebUserApproval (the make_policy closure, warpgate-core/src/config_providers/db.rs:421-433). The SSH side gets a keyboard-interactive prompt with a login URL (warpgate-protocol-ssh/src/server/session.rs:135-149), the user authenticates via real SSO in a browser, and that approval POSTs back to /auth/state/:id/approve (warpgate-protocol-http/src/api/auth.rs:341-383), matched by the auth-state ID plus a case-insensitive username check, and calls state.add_web_user_approval() on the SSH session's AuthState.

The gap in what exists today: none of the verified identity crosses that boundary, only a bare approval signal does. Closing it requires three structural changes, not a small patch:

1. The approve handler has no access to the approving browser's SsoLoginState / access token today, no Session extractor is wired in, and cross-node approvals proxy only the JSON request body between nodes (warpgate-admin/src/api/cluster_proxy.rs:129-149), so the token has to be added to that request payload to survive the hop.
2. AuthState (warpgate-common/src/auth/state.rs:128-142) has no field to hold verified identity material; one has to be added.
3. add_web_user_approval() (state.rs:262-265) currently takes no arguments and pushes a unit-variant credential; it has to accept and carry the verified claims through.

Vault-signing side. The change inside warpgate-vault itself is smaller and self-contained. VaultClient today has exactly one VaultAuth (Kubernetes/AppRole/AWS/Azure/GCP) fixed at construction, cached in a single shared Mutex<Option<CachedToken>> (warpgate-vault/src/client.rs:174-178); no JWT auth variant and no auth/jwt/login path exists yet. Adding a one-shot login_with_jwt() sibling to the existing login(), plus a sign_with_token() split off of sign_once that bypasses the shared cache entirely, is a contained addition reusing the existing Zeroizing plumbing and the JwtLogin request struct that already exists for the Kubernetes/GCP paths. One thing to account for: the existing 403-retry logic invalidates the shared cache by token_id, and has to explicitly skip that on the one-shot path so it doesn't touch the shared workload-identity token. If the RFC 8693 exchange path is the one built, this is also where it would live: exchange the access token for a Vault-audienced one before this call, rather than presenting the access token to Vault directly.

The call site (warpgate-protocol-ssh/src/client/mod.rs, where sign_ssh_key gets called) only has services and a session id available today, with no user token reachable from there. That's the same gap as the three items above, not a separate cost. Carrying the verified token from the SSO-approval boundary through to where the certificate actually gets signed is the bulk of the work here, not the Vault-side addition.

Vault-side config. Enable auth/jwt, point it at the same IdP's JWKS Warpgate's own SSO config already trusts, user_claim=sub rather than email, since sub is the IdP's stable identifier and emails get reassigned. bound_audiences is the specific setting #2300's review flagged as the anti-pattern when pointed at Warpgate's own client ID; if the access token's own aud is Vault or unset, that's fine as-is, if it's Warpgate's client ID, that's the same problem #2300 hit and argues for the RFC 8693 exchange step instead of setting bound_audiences to accept it. Tie the resulting entity into the same identity-templating mechanism from the direction above, default_user_template rendering the principal from identity.entity.metadata. That covers both halves together: who is asserting the identity, now a signature Vault checks itself, and what principal gets rendered from it.

What this design actually closes

A compromised Warpgate process can no longer mint a certificate for an arbitrary user by asserting it. It would need a live, unexpired, correctly-audienced token for that specific user, which it only has if that user is mid-session at that moment. That's a smaller and self-expiring blast radius than "any role the shared workload credential permits, indefinitely."

What this design doesn't close

That's a narrowing, not a close. Verifying a token's signature proves it's authentic, not that it's being used for the operation it was actually obtained for — the binding between "this verified token" and "this specific signing request" is still made by Warpgate's own bookkeeping (warpgate-protocol-http/src/api/auth.rs:341-383), in the same process this threat model assumes could be compromised. A compromised Warpgate doesn't need to forge anything: it can take a real, live token from any concurrently-approving user's session and wire it to a different, attacker-chosen signing request than the one that user actually approved.

Plausible fixes checked, and why none of them close it

Checked whether anything closes this specifically, not just narrows it further — short answer, no, for closely related reasons in most cases: Warpgate constructs every request the token-scoping mechanisms below would rely on, and the non-protocol approaches (attestation, capability tokens) fail for adjacent reasons of their own.

• RFC 8693 (Token Exchange) — fixes token audience, not operation scope. Different problem, doesn't apply here.
• RAR / RFC 9396 (Rich Authorization Requests) — closest shape in principle, but Warpgate is the OAuth client constructing the request (SsoClient::start_login, warpgate-sso/src/sso.rs), and the IdP never observes the SSH connection to validate it against anything real — RFC 9396 lets the AS enrich or replace authorization_details from its own data (§7.1), but only from facts the IdP already holds about the user, never anything about the SSH connection Warpgate is separately managing. A Vault check would just confirm Warpgate's later request matches what Warpgate itself told the IdP earlier — self-consistency, not verification. Partial benefit: forces a fresh, IdP-logged round-trip per distinct role, so a compromised Warpgate can't silently reuse one consent for a different role — detective, not preventive.
• DPoP (RFC 9449) / mutual TLS sender-constraining — proves who holds a token, not what it's used for. The spec's own text concedes it provides no guarantee once an adversary can run code in the client's execution context, which is this threat model exactly.
• OAuth Transaction Tokens (draft-ietf-oauth-transaction-tokens) — same failure as RAR, one hop later: a Transaction Token Service can apply its own policy to whatever request_context/request_details Warpgate supplies, but the draft puts that authorization policy explicitly out of scope, and any policy it did apply would still be evaluating Warpgate's account of a connection the service can't see.
• Step-up authentication (RFC 9470) — no field for operation-specific content, only authentication strength/recency. Shrinks the replay window; doesn't close anything.
• HTTP Message Signatures (RFC 9421) — proves who authored a request, not whether what they authored is honest. Same category as DPoP.
• Grant Management for OAuth 2.0 (FAPI's "Lodging Intent" pattern), and CIBA's out-of-band binding_message (a short interlock cue, weaker still) — genuinely different in kind: the intent is one shared object both the user's consent screen and Vault's issuance decision read from, so Warpgate can't show the user one role and claim another without the user seeing the real one. Converts silent substitution into visible, attributable substitution — real, but still not a close, since Warpgate authors the intent either way.
• Remote attestation / TEEs — proves the code is unmodified, not that code designed to self-assert a role is honest about which one.
• Capability-token schemes (macaroons, biscuits) — still need someone to attenuate the capability correctly at mint time, which is Warpgate's job here regardless.

The one fix that actually works, in full, and what a full fix would need instead

The one mechanism that actually removes the self-assertion, rather than notarizing or exposing it, isn't on the list above at all — it's Vault/OpenBao configuration, not a protocol. What actually stops a compromised Warpgate from signing for a more-privileged SSH role than the connecting user's own is auth/jwt role-level Access Control List (ACL) policy plus bound_claims — not which mount or which bound_audiences value is used, which is easy to get wrong here. Give each SSH-role class (wg-dev, wg-prod, etc.) its own auth/jwt role with token_policies granting exactly one ssh/sign/<role> path, and bound_claims keyed to an IdP-asserted, server-verified claim Warpgate can't forge. That second half is load-bearing: the role parameter in a login request is caller-supplied with no entitlement check of its own, so without bound_claims, a compromised Warpgate can just request the more-privileged role for any token it holds — bound_audiences alone won't stop it, since every Warpgate-held token shares the same audience. OpenBao has a stronger option worth using instead of caller-chosen roles entirely: token_policies_template_claims derives the granted policy directly from a claim, removing Warpgate's ability to pick the role at all. bound_claims (the option that also works on plain Vault) narrows the blast radius the same way but still lets Warpgate pick which role to request, gated by whether the claim matches — real protection, but resting on correct per-role configuration rather than removing the choice structurally.

Separately, and lower-stakes: giving each role its own auth/jwt mount fixes a different, narrower bug — identity-alias metadata is keyed by mount and overwritten wholesale on every login, so roles sharing a mount and resolving the same user to the same alias can have one role's login silently rewrite the metadata a still-valid token from another role renders its principal against. That's a principal-correctness hazard, not an authorization bypass; the ACL/bound_claims mechanism above is what actually prevents wrong-role signing regardless of whether mounts are shared.

The honest ceiling on all of this: a compromised Warpgate still holds every currently-connected user's token, so even correctly configured it can reach the union of privileges of everyone connected right now, not one arbitrary role fleet-wide. bound_claims caps the blast radius; it doesn't eliminate it.

Closing that fully needs a per-connection, target-scoped token from the IdP itself, outside anything Vault configuration can provide — which is what Teleport's tsh flow and step-ca's OIDC provisioner both do instead: the client mints its own certificate directly against whatever verifies identity, so the intermediary never asserts a principal on anyone's behalf at all. Among systems that keep the intermediary in the credential path rather than removing it, AWS Security Token Service (STS)'s SourceIdentity is the closest real analogue — immutable through role-chaining, but AWS's own documentation states plainly that AWS doesn't control the value in the first place, the same self-assertion problem, just narrower. HashiCorp Boundary, architecturally the same shape as this design, doesn't solve it either — Vault sees one Boundary-held identity per credential store regardless of how many users or sessions flow through it. That's a materially different design from the one here, out of scope for this direction, but worth naming plainly rather than implying any of the above would eventually get there.

What's still not covered, even if all of this gets built

It does not close the gap universally. Only sessions that actually went through SSO approval get this. A deployment on pure local password/pubkey auth has no token to forward. Closing the gap for those means requiring SSO in the credential policy for any target using Certificate auth, an enforceable policy choice, not a code guarantee. Everyone else stays on the narrower identity-templating design above.

It does not close PII exposure either, independent of which token type gets used. #2309's own measurement found the IdP in that deployment puts the same PII (email, in sub) in the access token as the ID token, so switching token types closes the audience anti-pattern, not the PII question. That needs a pairwise or pseudonymous subject identifier at the IdP, outside Warpgate's control, if it matters here.

The existing approval-replay caching, where remembered approvals are reused within a grace window, also needs reconsidering here, since replaying an approval without a fresh SSO round-trip risks forwarding an already-stale token. That failure mode is closed rather than open: Vault's own auth/jwt/login independently re-checks exp and the signature regardless of what Warpgate's local approval cache believes.

Related discussion

A comment on #26 raised an adjacent concern from a different angle: prompting the connecting user interactively for their own Vault-facing token at connect time, specifically so Warpgate isn't the sole thing Vault has to trust (#26 (comment)). Different mechanism, interactive per-connection challenge versus identity-templated roles plus per-session scoped tokens derived from the session Warpgate already established, but the same underlying instinct, and worth reading alongside this.

Open questions

What the directions above don't resolve:

• Host-binding. Issued certs would still be valid at every host trusting the CA, not just the one being connected to. Unclear whether a source-address constraint is the right answer or something else is.
• Revocation/CRL. The short TTL would be the only bound; no KRL distribution story exists yet.
• Per-entity rate limiting. Unavailable in both Vault and OpenBao's OSS tier (group_by on quotas is Enterprise-only in Vault, absent entirely in OpenBao), an upstream limitation either way.
• Plaintext token storage, and PII in the token regardless of type. Both the ID token and, per feat(http): forward the SSO access token to HTTP targets on request #2309, the access token get stored the same way, DB-backed in http_sessions, unencrypted at rest today. Switching from ID token to access token doesn't change this, and per feat(http): forward the SSO access token to HTTP targets on request #2309's own measurement doesn't reduce PII exposure either, since the same PII often ends up in both token types for a given IdP. Whether encryption-at-rest for http_sessions gets addressed first, or is acceptable given short IdP token lifetimes, is unresolved.

And the more basic open question: is either of these the direction #26 should settle on at all, or is there a better answer to "how does Vault know who's really asking" that this doesn't consider?