Repository navigation
Conversation
…outs When a websocket is proxied through queue-proxy or the activator, httputil.ReverseProxy hijacks the connection and writes the 101 response and all subsequent frames directly to it, bypassing the timeoutWriter. lastWriteTime therefore stayed zero, so the response start and idle timeouts fired on active websockets, cancelling the request context and closing the connection. Wrap the hijacked connection (and the buffered writer returned with it) so writes update lastWriteTime, and skip writing an error response to a hijacked connection when a timeout fires. Fixes knative#16169 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: mslynch The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
|
Welcome @mslynch! It looks like this is your first PR to knative/serving 🎉 |
|
Hi @mslynch. Thanks for your PR. I'm waiting for a knative member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Only bytes written to the hijacked net.Conn count, not the 101 Switching Protocols response that ReverseProxy writes to the hijacked buffered writer. The response start timeout therefore measures the time until the first bytes after the upgrade, and the existing TestWebSocketWithTimeout expectations are unchanged. Also clear the timer pool in the new unit tests, since other tests leave fake clock timers in it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes #16169
When a websocket is proxied through queue-proxy (or the activator),
httputil.ReverseProxyhijacks the connection and writes the101 Switching Protocolsresponse and every subsequent frame directly to the hijackednet.Conn/bufio.ReadWriter, bypassingtimeoutWriter.Write/WriteHeader.lastWriteTimetherefore stays zero for the life of the websocket, so:When either fires,
ServeHTTPreturns, the request context is cancelled, andReverseProxycloses the backend connection, terminating an active websocket. With the defaultrevision-response-start-timeout-secondsof 300, every websocket is cut at exactly 5 minutes regardless of traffic. queue-proxy also logshttp: response.WriteHeader on hijacked connectionas it tries to write the 504.Proposed Changes
Hijackreturns a wrappednet.Connwhose writes updatelastWriteTime, so bytes sent by the app after the upgrade count towards the response start and idle timeouts. As for regular HTTP responses, only bytes from the app count towards the idle timeout.http.ErrHandlerTimeout.NewTimeoutHandler+ReverseProxy: active connections outlive the response start and idle timeouts (both fail onmain), and silent connections are still closed by both timeouts.TestWebSocketActiveConnectionOutlivesTimeouts: exchanges a message every second for 30s against revisions with a 10s response start / idle timeout. Fails onmainat ~12s withclose 1006 (abnormal closure); passes with this change.timeoutSecondsstill bounds the total lifetime of the websocket.Validated on a kind cluster with Kourier: all
TestWebSocket*e2e tests pass.Decision needed: does the 101 count as the response starting?
This PR currently does not count the
101 Switching Protocolsresponse towards the response start timeout.ReverseProxywrites the 101 to the hijackedbufio.ReadWriter, which is left unwrapped, so the response start timeout measures the time until the app's first bytes after the upgrade. That keeps the existingTestWebSocketWithTimeoutcase "exceeds response start timeout seconds" passing: the server upgrades, then waits 20s before echoing, and the connection is closed by the 10s response start timeout.The alternative is to also point the buffered writer at the wrapped connection (
rw.Writer.Reset(wrappedConn)), so the upgrade response itself starts the response. Arguments for it:responseStartTimeoutSecondsas how long to wait for the container "to begin sending any network traffic", and the 101 comes from the container.With that alternative, a websocket whose app is silent after the handshake would only be bounded by the idle timeout and
timeoutSeconds. The existing "exceeds response start timeout seconds" e2e case would need to flip to expect success. Either choice fixes active websockets being closed; this only affects websockets that are silent after the upgrade. Happy to switch to whichever behaviour maintainers prefer.Release Note
🤖 Generated with Claude Code