Skip to content

Decide redirects in one place instead of along a loop - #562

Merged
alistair3149 merged 8 commits into
masterfrom
fix/redirect-algorithm
Aug 11, 2026
Merged

Decide redirects in one place instead of along a loop#562
alistair3149 merged 8 commits into
masterfrom
fix/redirect-algorithm

Conversation

@alistair3149

Copy link
Copy Markdown
Member

Fixes #549
Fixes #555
Fixes #556
Fixes #559

Supersedes #551, and is built on its branch so its verified work is carried rather than re-derived.

fetchCore hand-rolled the Fetch standard's redirect algorithm and implemented part of it. Each rule it got wrong was a separate open issue in the same fifteen lines, and none could be tested without standing up a server — which is why three of the four went unnoticed until a socket count was taken.

The rules now live in src/transport/requestChain.ts as one pure decision. nextHop reads what was sent and the status and Location that came back, and answers deliver, follow or refuse; fetchCore performs the I/O each answer asks for. Because a refusal is a returned value rather than a throw, every response that is not delivered leaves through one place — which is what disposes of the abandoned bodies, instead of a call at each exit that a later edit can forget.

What the rules now say

Rule Issue
Follow only 301, 302, 303, 307 and 308 — a 300 offers a choice and a 305 names a proxy, so neither is an address #559
Refuse a hop that would drop the request body #549
Refuse a hop from https to http #556
Drop credential headers across any change of origin #556
Dispose of every abandoned body, on the followed hop as well as at each refusal #555
Read a failing source's diagnostics up to 8 KB rather than whole #559

Two of these are deliberate departures from the standard, both because this server fetches URLs a caller chose: the standard re-sends a 301/302/303 as a bodyless GET, and it follows an httpshttp hop once it has stripped the credentials.

Credentials go on a change of origin, not of host. That is stricter than node-fetch, whose rule keeps them for a subdomain of the current host and ignores the port — on a host that gives its tenants subdomains, that would hand one tenant's credentials to another.

The method is now derived from the presence of a body, so fetchCore has no method option and a bodyless POST is not representable. That retires the question #551 left open — whether the refusal should key on the body or the method — by making the two the same thing.

What this costs, measured

Unchanged from #551, and still the thing to weigh: a wiki publishing a 301/302/303-redirecting query-service URL loses queries that worked before. Verified live:

Published endpoint Answers a form POST with Target answers the same POST
http://database.factgrid.de/sparql 301https://database.factgrid.de/sparql 200
http://dbpedia.org/sparql 303https://dbpedia.org/sparql 200

Both are plaintext endpoints upgrading to TLS, so the first request already went out in the clear. Refusing surfaces that; following it hides it. Endpoints answering 307/308 are unaffected.

For the reviewer

  • Extracting the decision, rather than finishing the loop in place. Three independent designs were compared and judges split 2–1 for extraction, so this is a close call worth your own view. What tipped it: two rules in this exact loop were found unpinned by any test during the work on Send a bodyless GET after a 301, 302 or 303 redirect #551 — one header set three-quarters unexercised, one disposal call with no test at all — so "each rule is a millisecond value assertion" is not hypothetical here. The counter-argument is that 55 lines with one caller does not need a module.
  • requestChain.ts imports nothing. No node-fetch, no DNS, no clock. That is what makes the rules testable, and it is also what would let an eventual move to undici keep this file and delete only the parts that duplicate the standard.
  • An insecure hop is reported ahead of a private-address one. The pure rules run before the transport's DNS check, since they cost nothing and it costs a lookup, so httpshttp://169.254.169.254 is reported as insecure rather than as private. Both refuse it. One existing SSRF test's fixture was an httpshttp redirect and so was testing two rules at once; it now uses an https target, which is what it meant to test.

Verification

tests/transport/requestChain.test.ts — 39 cases, no mocks, no sockets, no Response objects. 14 mutations applied one at a time, all 14 caught. The one that initially survived was a real gap: resolving a relative Location against the start of the chain rather than the current hop passed everything, because every relative-Location case was on the first hop where the two URLs are equal. There is now a second-hop case.

tests/transport/httpFetch.disposal.test.ts gained the test #555 needed and had no way to express: a followed hop whose body is never read. It fails when disposal is moved back to the refusal path only. Its harness now records every socket the server served, because the second hop used to overwrite the first and hide exactly the leak being measured.

Gates individually, all exit 0: npm run lint, npm run typecheck, npm run fmt:check, npm test (151 files, 2122 tests).

Not in this change

#561 (a token in a redirect target escaping the endpoint redaction) is in the Wikibase layer, not here. The uncapped success body — fetchPageHtml reads an unbounded text() on the discovery and probe paths — is a different resource on a different path. Duplicate Location headers join to a bogus URL that this still follows; it needs its own decision about which to honour.

AI-authored — Claude Code, Opus 5 (xhigh); the maintainer asked for one holistic fix after a review found four issues sharing a root cause, and chose to fold #551 in rather than merge it first; diff not yet human-reviewed; 14 mutations verified caught, the disposal test verified to fail without the fix, the two live endpoint measurements reproduced with curl, gates green locally.

alistair3149 and others added 7 commits August 11, 2026 12:38
The redirect loop in fetchCore re-sent the original method and body on
every hop, so a POST redirected by a 301, 302 or 303 arrived at the
target as a POST carrying a body the target did not expect, and was
rejected. Only 307 and 308 ask for the method and body again.

The loop now tracks the method, body and headers of the request it is
about to send rather than reading them from the caller's options each
hop. A status other than 307 or 308 turns the next request into a GET
with no body and drops the headers that describe a body, so the form
content type does not travel on a request that no longer carries a form.
Because the downgrade is carried forward, a 302 followed by a 307 keeps
the GET instead of resurrecting the POST.

Reachable today through a configured sparqlEndpoint behind a redirect,
such as an http URL or a bare host: postForm is the only caller that
sends a body.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Only Content-Type was covered, so trimming the dropped-header set to that
one alone left the suite green. The new test sends Content-Encoding,
Content-Language and Content-Location as caller headers and asserts none
of them reach the downgraded hop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
postForm and its only caller arrived with the Wikibase pack in the same
unreleased cycle as this fix, so no released version sends a body
through the redirect loop. The entry would tell a reader upgrading from
0.16.0 about a failure that version cannot produce.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
node-fetch recomputes Content-Length only for a request that carries a
body, so a value the caller set reaches the bodyless GET a redirect
downgrade produces, telling the target to expect a body that never
arrives. The Fetch standard leaves the header out of its request-body
list because there the fetch layer always owns it; here it does not, so
the downgrade removes it alongside the other body-describing headers.

Also names the downgrade condition rather than testing the preserving
set in the negative.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A 301, 302 or 303 asks for the request to be re-sent as a GET with no
body. For the one caller that sends a body that is not a smaller request,
it is a different one: a SPARQL query travels in the body, so the GET
arrives with nothing to run and the service answers about a request the
caller never made. Following the redirect and reporting that answer hides
the actual fault, which is that the published endpoint URL has moved.

Such a hop is now refused. 307 and 308 re-send the method and body as
before, and a request with no body — which is every other request this
module sends, all of them already GETs — follows any redirect unchanged.
The 3xx body is destroyed on the way out, so the refusal does not strand
the connection it declined to follow.

The Wikibase layer turns the refusal into a message naming the setting to
change, and names the target by scheme and host only. A query-service URL
can carry a token in its path or query, which is why that layer
substitutes the endpoint out of anything it reports; a target derived from
that URL inherits the token and, differing in scheme, is not a substring
that substitution can find. Scheme and host identify the redirect without
carrying a credential.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The redirect loop hand-rolled the Fetch standard's redirect algorithm and
implemented part of it, so each rule it got wrong was a separate bug in
the same fifteen lines, and none of them could be tested without standing
up a server. Four of them were open at once: a body silently discarded, a
connection held for every hop followed, credentials re-sent across a
change of host, and any 3xx with a Location treated as a destination.

The rules now live in requestChain.ts as one pure decision. nextHop reads
what was sent and the status and Location that came back, and answers
deliver, follow or refuse; fetchCore performs the I/O each answer asks
for. Because a refusal is a returned value rather than a throw, every
response that is not delivered leaves through one place, which is what
disposes of the abandoned bodies rather than a call at each exit that a
later edit can forget.

What the rules now say: follow only 301, 302, 303, 307 and 308; refuse a
hop that would drop the request body; refuse a hop from https to http;
drop credential headers across any change of origin; cap the chain; and
read a failing source's diagnostics up to a limit rather than whole.

Credentials go on a change of origin rather than of host, which is
stricter than node-fetch, whose rule keeps them for a subdomain and
ignores the port — on a host that gives its tenants subdomains that would
hand one tenant's credentials to another.

The method is derived from the presence of a body, so a bodyless POST is
not representable and the refusal cannot be reached by a request that has
nothing to lose.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rationale for each rule belongs in the commit and the pull request, not
beside the rule, and a doc comment that walks through guard clauses the
reader can see earns nothing. What is left is the part that cannot be
derived: the two places this departs from the Fetch standard, why
credentials go on a change of origin rather than of host, why a refusal
withholds the target, and the ordering against the transport's own
address check.
@alistair3149
alistair3149 marked this pull request as ready for review August 11, 2026 20:38
@alistair3149
alistair3149 merged commit 9fefe68 into master Aug 11, 2026
2 checks passed
@alistair3149
alistair3149 deleted the fix/redirect-algorithm branch August 11, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment