Skip to content

fix(server): do not wait for response.write for response.end #2383

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
May 9, 2025

Conversation

ardatan
Copy link
Owner

@ardatan ardatan commented May 5, 2025

Some implementations like compression npm package do not implement response.write(data, callback) signature, but whatwg-node/server waits for it to finish the response stream.
Then it causes the response stream hangs when the compression package takes the stream over when the response data is larger than its threshold.

It is actually a bug in compression package;
expressjs/compression#46
But since it is a common mistake, we prefer to workaround this on our end.

Now after calling response.write, it no longer uses callback but first it checks the result;

if it is true, it means stream is drained and we can call response.end immediately.
else if it is false, it means the stream is not drained yet, so we can wait for the drain event to call response.end.

This PR also does some improvements by unifying the "write and wait for drain logic".
And it removes the nodeRequestResponseMap stuff for performance gain.

Fixes graphql-hive/graphql-yoga#3997

Copy link
Contributor

coderabbitai bot commented May 5, 2025

📝 Walkthrough

Summary by CodeRabbit

  • Bug Fixes
    • Implemented a workaround to prevent response stream hanging when using certain compression middleware, ensuring responses complete reliably.
  • Refactor
    • Streamlined and unified internal stream handling for better reliability and maintainability.
  • New Features
    • Improved multipart form data streaming for more efficient and incremental processing.
  • Tests
    • Added comprehensive tests for large request bodies with compression middleware.
    • Enhanced test descriptions and coverage for promise handling utilities.
  • Chores
    • Updated and pinned several development dependencies for improved stability.
    • Introduced a new example test server script for easier local testing.

Walkthrough

This update refactors stream and response writing logic across several packages to use new utility functions for safe, backpressure-aware writing and controlled stream termination. It addresses issues with Express's compression middleware by ensuring responses do not hang. Related tests and documentation are updated, and several dependencies are revised.

Changes

Files/Paths Change Summary
packages/server/src/utils.ts, packages/node-fetch/src/utils.ts Added safeWrite and endStream utility functions for safe, backpressure-aware writing and controlled stream termination.
packages/server/src/utils.ts, packages/node-fetch/src/FormData.ts,
packages/node-fetch/src/WritableStream.ts, packages/node-fetch/src/TransformStream.ts, packages/node-fetch/src/fetchNodeHttp.ts
Refactored to use new utilities for stream writing and ending; replaced manual promise/callback handling with unified helpers.
packages/server/src/createServerAdapter.ts Removed usage of nodeRequestResponseMap; normalized requests now receive the response as an argument.
packages/server/test/reproductions.spec.ts, test.mjs Added/updated tests to verify Express with compression middleware and a minimal server adapter use case.
packages/server/test/test-server.ts Changed Fastify test server startup to construct the server URL after listening, using the actual port.
packages/server/package.json Upgraded and pinned several devDependencies; added compression and its types.
.changeset/whole-planes-mate.md Added a changelog note describing the workaround for the compression middleware issue.
packages/promise-helpers/tests/handleMaybePromise.spec.ts Reformatted falsy value tests for clarity; no logic changes.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant ExpressApp
    participant CompressionMiddleware
    participant ServerAdapter
    participant Response

    Client->>ExpressApp: POST /my-path (large body)
    ExpressApp->>CompressionMiddleware: Pass request
    CompressionMiddleware->>ServerAdapter: Pass request/response
    ServerAdapter->>Response: Write response body using safeWrite
    alt write() returns true
        ServerAdapter->>Response: end() immediately
    else write() returns false
        ServerAdapter->>Response: Wait for 'drain', then end()
    end
    Response-->>Client: Compressed JSON response
Loading

Assessment against linked issues

Objective Addressed Explanation
Prevent hanging responses when using Express's compression middleware with @whatwg-node/server (#3997)
Ensure correct response delivery for all request sizes with compression enabled (#3997)
No regression in normal (non-compression) response handling (#3997)

Possibly related PRs

Poem

In the warren of streams, where responses might stall,
A rabbit hopped in and rewrote it all.
With safeWrite and endStream, the bytes smoothly flow,
No more hanging with compression—just go!
So servers and clients can hop with delight,
Thanks to a bunny who debugged through the night.
🐇✨

Tip

⚡️ Faster reviews with caching
  • CodeRabbit now supports caching for code and dependencies, helping speed up reviews. This means quicker feedback, reduced wait times, and a smoother review experience overall. Cached data is encrypted and stored securely. This feature will be automatically enabled for all accounts on May 16th. To opt out, configure Review - Disable Cache at either the organization or repository level. If you prefer to disable all data retention across your organization, simply turn off the Data Retention setting under your Organization Settings.

Enjoy the performance boost—your workflow just got faster.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 39a5cc4 and 1e764d6.

📒 Files selected for processing (1)
  • packages/server/package.json (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/server/package.json
⏰ Context from checks skipped due to timeout of 90000ms (17)
  • GitHub Check: server (ponyfill)
  • GitHub Check: server (native)
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: unit / node 23
  • GitHub Check: server (undici)
  • GitHub Check: node-fetch (noConsumeBody)
  • GitHub Check: e2e / cloudflare-workers
  • GitHub Check: unit / node 22
  • GitHub Check: unit / node 18
  • GitHub Check: unit / node 20
  • GitHub Check: unit / deno
  • GitHub Check: e2e / cloudflare-modules
  • GitHub Check: e2e / azure-function
  • GitHub Check: esm
  • GitHub Check: unit / bun
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: prettier

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

github-actions bot commented May 5, 2025

🚀 Snapshot Release (alpha)

The latest changes of this PR are available as alpha on npm (based on the declared changesets):

Package Version Info
@whatwg-node/fetch 0.10.7-alpha-20250509160231-1e764d6503d33d7a355917720b97a4de05039165 npm ↗︎ unpkg ↗︎
@whatwg-node/node-fetch 0.7.19-alpha-20250509160231-1e764d6503d33d7a355917720b97a4de05039165 npm ↗︎ unpkg ↗︎
@whatwg-node/server 0.10.6-alpha-20250509160231-1e764d6503d33d7a355917720b97a4de05039165 npm ↗︎ unpkg ↗︎

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.changeset/whole-planes-mate.md (1)

1-11: Well-documented changeset with minor formatting improvements needed

The changeset correctly describes the problem and solution. Consider these small improvements:

  1. Format the bare URL as a markdown link
  2. Fix the usage of "workaround" as a verb
 Some implementations like `compression` npm package do not implement `response.write(data, callback)` signature, but whatwg-node/server waits for it to finish the response stream.
 It is actually a bug in `compression` package;
-https://github.com/expressjs/compression/blob/master/index.js#L99
-But since it is a common mistake, we prefer to workaround this on our end.
+[See code here](https://github.com/expressjs/compression/blob/master/index.js#L99)
+But since it is a common mistake, we prefer to work around this on our end.
🧰 Tools
🪛 LanguageTool

[grammar] ~11-~11: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end.

(NOUN_VERB_CONFUSION)

🪛 markdownlint-cli2 (0.17.2)

10-10: Bare URL used
null

(MD034, no-bare-urls)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ed60ece and f96bef2.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (4)
  • .changeset/whole-planes-mate.md (1 hunks)
  • packages/server/package.json (1 hunks)
  • packages/server/src/utils.ts (1 hunks)
  • packages/server/test/reproductions.spec.ts (2 hunks)
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[grammar] ~11-~11: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end.

(NOUN_VERB_CONFUSION)

🪛 markdownlint-cli2 (0.17.2)
.changeset/whole-planes-mate.md

10-10: Bare URL used
null

(MD034, no-bare-urls)

🪛 Biome (1.9.4)
packages/server/test/reproductions.spec.ts

[error] 84-84: Don't focus the test.

The 'only' method is often used for debugging or during implementation. It should be removed before deploying to production.
Consider removing 'only' to ensure all tests are executed.
Unsafe fix: Remove focus from test.

(lint/suspicious/noFocusedTests)

⏰ Context from checks skipped due to timeout of 90000ms (21)
  • GitHub Check: e2e / cloudflare-modules
  • GitHub Check: e2e / cloudflare-workers
  • GitHub Check: e2e / azure-function
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / deno
  • GitHub Check: unit / bun
  • GitHub Check: unit / node 23
  • GitHub Check: unit / node 22
  • GitHub Check: alpha / snapshot
  • GitHub Check: unit / node 20
  • GitHub Check: server (undici)
  • GitHub Check: type check
  • GitHub Check: server (ponyfill)
  • GitHub Check: esm
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: unit / node 18
  • GitHub Check: server (native)
  • GitHub Check: lint
  • GitHub Check: node-fetch (noConsumeBody)
  • GitHub Check: prettier
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (3)
packages/server/package.json (1)

45-45: Dependencies added correctly for testing with compression

The addition of compression and @types/compression as devDependencies is appropriate for testing the fix. These dependencies will allow the test suite to verify that the server adapter works correctly with Express's compression middleware.

Also applies to: 49-49

packages/server/src/utils.ts (1)

358-359: Good fix for compatibility with middleware packages

This change addresses the core issue by calling endResponse immediately after serverResponse.write, rather than waiting for a write callback that may never be invoked. This ensures compatibility with packages like compression that don't properly implement the response.write(data, callback) signature.

The change is minimal and focused only on the optimized node-fetch buffer path, which is exactly what was needed to fix the issue.

packages/server/test/reproductions.spec.ts (1)

3-3: Import added for the compression middleware

The import statement for the compression middleware is correctly added to support the new test case.

Copy link
Contributor

github-actions bot commented May 5, 2025

@benchmarks/server results (native)

     ✓ no-errors
     ✓ expected-result

   ✓ checks.........................: 100.00% ✓ 233026      ✗ 0     
     data_received..................: 23 MB   781 kB/s
     data_sent......................: 9.3 MB  311 kB/s
     http_req_blocked...............: avg=1.4µs    min=872ns    med=1.18µs   max=272.44µs p(90)=1.89µs   p(95)=2.06µs  
     http_req_connecting............: avg=1ns      min=0s       med=0s       max=120.03µs p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=194.79µs min=143.25µs med=184.2µs  max=8.4ms    p(90)=211.28µs p(95)=221.03µs
       { expected_response:true }...: avg=194.79µs min=143.25µs med=184.2µs  max=8.4ms    p(90)=211.28µs p(95)=221.03µs
     http_req_failed................: 0.00%   ✓ 0           ✗ 116513
     http_req_receiving.............: avg=25.19µs  min=13.45µs  med=23.04µs  max=2.58ms   p(90)=30.91µs  p(95)=33.69µs 
     http_req_sending...............: avg=6.35µs   min=4.19µs   med=5.49µs   max=404.7µs  p(90)=8.15µs   p(95)=8.77µs  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=163.24µs min=119.08µs med=152.9µs  max=8.33ms   p(90)=176.25µs p(95)=185.26µs
     http_reqs......................: 116513  3883.339272/s
     iteration_duration.............: avg=253.01µs min=186.68µs med=241.57µs max=9.59ms   p(90)=272.15µs p(95)=284.73µs
     iterations.....................: 116513  3883.339272/s
     vus............................: 1       min=1         max=1   
     vus_max........................: 1       min=1         max=1   

Copy link
Contributor

github-actions bot commented May 5, 2025

@benchmarks/node-fetch results (noConsumeBody)

   ✓ active_handles.................: avg=140.91909 min=13     med=142     max=187     p(90)=160     p(95)=163    
     data_received..................: 21 MB  686 kB/s
     data_sent......................: 13 MB  444 kB/s
     http_req_blocked...............: avg=3.83µs    min=611ns  med=1.48µs  max=6.66ms  p(90)=2.14µs  p(95)=2.52µs 
     http_req_connecting............: avg=1.75µs    min=0s     med=0s      max=6.63ms  p(90)=0s      p(95)=0s     
     http_req_duration..............: avg=22.22ms   min=2.23ms med=21.6ms  max=1.03s   p(90)=27.75ms p(95)=29.7ms 
       { expected_response:true }...: avg=22.22ms   min=2.23ms med=21.6ms  max=1.03s   p(90)=27.75ms p(95)=29.7ms 
     http_req_failed................: 0.00%  ✓ 0           ✗ 134474
     http_req_receiving.............: avg=34.56µs   min=9.65µs med=25.38µs max=18.72ms p(90)=40.14µs p(95)=46.86µs
     http_req_sending...............: avg=12.28µs   min=3.41µs med=6.98µs  max=10.18ms p(90)=10.63µs p(95)=14.77µs
     http_req_tls_handshaking.......: avg=0s        min=0s     med=0s      max=0s      p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=22.18ms   min=2.17ms med=21.56ms max=1.03s   p(90)=27.7ms  p(95)=29.64ms
     http_reqs......................: 134474 4481.917261/s
     iteration_duration.............: avg=44.57ms   min=9.99ms med=43.03ms max=1.07s   p(90)=48.5ms  p(95)=54.52ms
     iterations.....................: 67223  2240.492021/s
     vus............................: 100    min=100       max=100 
     vus_max........................: 100    min=100       max=100 

Copy link
Contributor

github-actions bot commented May 5, 2025

@benchmarks/server results (ponyfill)

     ✓ no-errors
     ✓ expected-result

   ✓ checks.........................: 100.00% ✓ 298490      ✗ 0     
     data_received..................: 29 MB   980 kB/s
     data_sent......................: 12 MB   398 kB/s
     http_req_blocked...............: avg=1.42µs   min=911ns    med=1.2µs    max=286.11µs p(90)=1.87µs   p(95)=2.03µs  
     http_req_connecting............: avg=0ns      min=0s       med=0s       max=103.62µs p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=138.05µs min=83.22µs  med=133.21µs max=6.94ms   p(90)=156.34µs p(95)=163.5µs 
       { expected_response:true }...: avg=138.05µs min=83.22µs  med=133.21µs max=6.94ms   p(90)=156.34µs p(95)=163.5µs 
     http_req_failed................: 0.00%   ✓ 0           ✗ 149245
     http_req_receiving.............: avg=24.56µs  min=11.09µs  med=22.71µs  max=2.73ms   p(90)=30.6µs   p(95)=33.18µs 
     http_req_sending...............: avg=6.28µs   min=4µs      med=5.41µs   max=322.58µs p(90)=8.08µs   p(95)=8.78µs  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=107.2µs  min=63.72µs  med=102.13µs max=6.84ms   p(90)=121.51µs p(95)=127.5µs 
     http_reqs......................: 149245  4974.602326/s
     iteration_duration.............: avg=196.58µs min=138.35µs med=191.27µs max=7.07ms   p(90)=217.58µs p(95)=226.92µs
     iterations.....................: 149245  4974.602326/s
     vus............................: 1       min=1         max=1   
     vus_max........................: 1       min=1         max=1   

Copy link
Contributor

github-actions bot commented May 5, 2025

@benchmarks/server results (undici)

     ✓ no-errors
     ✓ expected-result

   ✓ checks.........................: 100.00% ✓ 215866      ✗ 0     
     data_received..................: 22 MB   723 kB/s
     data_sent......................: 8.6 MB  288 kB/s
     http_req_blocked...............: avg=1.44µs   min=932ns    med=1.22µs   max=254.32µs p(90)=1.9µs    p(95)=2.07µs  
     http_req_connecting............: avg=1ns      min=0s       med=0s       max=120.98µs p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=214.13µs min=160.51µs med=202.24µs max=57.55ms  p(90)=229.06µs p(95)=238.58µs
       { expected_response:true }...: avg=214.13µs min=160.51µs med=202.24µs max=57.55ms  p(90)=229.06µs p(95)=238.58µs
     http_req_failed................: 0.00%   ✓ 0           ✗ 107933
     http_req_receiving.............: avg=25.98µs  min=14.23µs  med=23.7µs   max=3ms      p(90)=31.82µs  p(95)=34.26µs 
     http_req_sending...............: avg=6.49µs   min=4.14µs   med=5.77µs   max=292.84µs p(90)=8.22µs   p(95)=8.96µs  
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=181.65µs min=133.1µs  med=169.92µs max=57.48ms  p(90)=193.11µs p(95)=201.72µs
     http_reqs......................: 107933  3597.632121/s
     iteration_duration.............: avg=273.36µs min=211.65µs med=260.18µs max=57.68ms  p(90)=290.99µs p(95)=302.77µs
     iterations.....................: 107933  3597.632121/s
     vus............................: 1       min=1         max=1   
     vus_max........................: 1       min=1         max=1   

Copy link
Contributor

github-actions bot commented May 5, 2025

@benchmarks/node-fetch results (consumeBody)

   ✓ active_handles.................: avg=140.48708 min=53      med=142     max=192      p(90)=160     p(95)=164    
     data_received..................: 22 MB  715 kB/s
     data_sent......................: 14 MB  458 kB/s
     http_req_blocked...............: avg=4.11µs    min=590ns   med=1.11µs  max=11.42ms  p(90)=1.9µs   p(95)=2.11µs 
     http_req_connecting............: avg=2.45µs    min=0s      med=0s      max=11.39ms  p(90)=0s      p(95)=0s     
     http_req_duration..............: avg=21.33ms   min=7.44ms  med=20.79ms max=853.73ms p(90)=26.91ms p(95)=28.55ms
       { expected_response:true }...: avg=21.33ms   min=7.44ms  med=20.79ms max=853.73ms p(90)=26.91ms p(95)=28.55ms
     http_req_failed................: 0.00%  ✓ 0           ✗ 140219
     http_req_receiving.............: avg=30.13µs   min=8.75µs  med=21.72µs max=16.59ms  p(90)=35.13µs p(95)=40.94µs
     http_req_sending...............: avg=10.79µs   min=2.92µs  med=5.18µs  max=39.78ms  p(90)=8.88µs  p(95)=11.67µs
     http_req_tls_handshaking.......: avg=0s        min=0s      med=0s      max=0s       p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=21.29ms   min=7.41ms  med=20.75ms max=853.65ms p(90)=26.87ms p(95)=28.5ms 
     http_reqs......................: 140219 4673.612214/s
     iteration_duration.............: avg=42.76ms   min=20.11ms med=41.24ms max=879.78ms p(90)=46.73ms p(95)=52.23ms
     iterations.....................: 70087  2336.056164/s
     vus............................: 100    min=100       max=100 
     vus_max........................: 100    min=100       max=100 

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
.changeset/whole-planes-mate.md (4)

5-6: Enhance clarity and punctuation in the problem description.

Consider refining the sentence for readability and adding a comma for correct apposition:

- Some implementations like `compression` npm package do not implement `response.write(data, callback)` signature, but whatwg-node/server waits for it to finish the response stream.
+ Some implementations, such as the `compression` npm package, do not implement the `response.write(data, callback)` signature, yet `@whatwg-node/server` currently waits for it before ending the response stream.

6-7: Correct verb form for consistency.

Replace “hangs” with the infinitive “hang” and streamline the clause:

- Then it causes the response stream hangs when the compression package takes the stream over when the response data is larger than its threshold.
+ This can cause the response stream to hang if `compression` takes over when streaming large responses.

8-10: Fix punctuation, verb usage, and format the URL as a markdown link.

  • Use a colon instead of a semicolon.
  • Change “workaround” (noun) to “work around” (verb).
  • Embed the bare URL in markdown for better readability.
- It is actually a bug in `compression` package;
- https://github.com/expressjs/compression/blob/master/index.js#L99
- But since it is a common mistake, we prefer to workaround this on our end.
+ This is actually a bug in the `compression` package:
+ [See the implementation here](https://github.com/expressjs/compression/blob/master/index.js#L99).
+ However, because this mistake is common, we prefer to work around it on our end.
🧰 Tools
🪛 LanguageTool

[grammar] ~10-~10: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

🪛 markdownlint-cli2 (0.17.2)

9-9: Bare URL used
null

(MD034, no-bare-urls)


12-12: Clarify the scope and context of the change.

Explicitly reference that this behavior is introduced by the patch:

- So now the server adapter calls `response.end` immediately after `response.write` for static responses.
+ This patch makes the server adapter call `response.end` immediately after `response.write` for static responses.
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2549b08 and 86aa384.

📒 Files selected for processing (1)
  • .changeset/whole-planes-mate.md (1 hunks)
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[grammar] ~10-~10: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

🪛 markdownlint-cli2 (0.17.2)
.changeset/whole-planes-mate.md

9-9: Bare URL used
null

(MD034, no-bare-urls)

⏰ Context from checks skipped due to timeout of 90000ms (20)
  • GitHub Check: e2e / cloudflare-modules
  • GitHub Check: e2e / cloudflare-workers
  • GitHub Check: unit / node 23
  • GitHub Check: e2e / azure-function
  • GitHub Check: unit / deno
  • GitHub Check: unit / node 22
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / bun
  • GitHub Check: unit / node 20
  • GitHub Check: unit / node 18
  • GitHub Check: type check
  • GitHub Check: prettier
  • GitHub Check: esm
  • GitHub Check: lint
  • GitHub Check: server (undici)
  • GitHub Check: server (ponyfill)
  • GitHub Check: server (native)
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: node-fetch (noConsumeBody)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (1)
.changeset/whole-planes-mate.md (1)

1-3: Frontmatter header follows the standard Changeset format and does not require updates.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
packages/server/src/utils.ts (1)

5-11: Import looks good but consider keeping the list alphabetised

handleMaybePromise is correctly added to the helper imports.
(Optional) To keep imports tidy and minimise merge conflicts down the line, you may want to alphabetise this import list (e.g. move handleMaybePromise after fakePromise).

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 86aa384 and 69830ee.

📒 Files selected for processing (1)
  • packages/server/src/utils.ts (4 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
packages/server/src/utils.ts (1)
packages/promise-helpers/src/index.ts (1)
  • handleMaybePromise (29-42)
⏰ Context from checks skipped due to timeout of 90000ms (18)
  • GitHub Check: unit / deno
  • GitHub Check: esm
  • GitHub Check: e2e / azure-function
  • GitHub Check: unit / node 23
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / node 22
  • GitHub Check: unit / node 20
  • GitHub Check: unit / bun
  • GitHub Check: server (undici)
  • GitHub Check: server (ponyfill)
  • GitHub Check: unit / node 18
  • GitHub Check: type check
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: lint
  • GitHub Check: server (native)
  • GitHub Check: Analyze (javascript-typescript)
  • GitHub Check: prettier
  • GitHub Check: node-fetch (noConsumeBody)
🔇 Additional comments (2)
packages/server/src/utils.ts (2)

366-369: 👍 Good switch to promise-based flow

Replacing the callback form of write with handleMaybePromise → safeWrite removes the dependency on middleware correctly passing a callback, fixing the compression issue without changing observable behaviour for well-behaved implementations.


384-386: 👍 Consistent treatment for Uint8Array bodies

The same promise-based pattern is applied to Uint8Array bodies, keeping the control flow uniform across buffer types.

Comment on lines +321 to +303
function safeWrite(chunk: any, serverResponse: NodeResponse) {
// @ts-expect-error http and http2 writes are actually compatible
const result = serverResponse.write(chunk);
if (!result) {
return new Promise(resolve => serverResponse.once('drain', resolve));
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

safeWrite can still hang when the socket closes before drain fires

safeWrite resolves only on the 'drain' event. If serverResponse.write returns false and the connection is closed or errors out before the internal buffer is drained, 'drain' never fires – the returned promise remains pending and endResponse is never called, resurrecting the very hang this PR tries to eliminate.

Recommended refinement:

-function safeWrite(chunk: any, serverResponse: NodeResponse) {
-  // @ts-expect-error http and http2 writes are actually compatible
-  const result = serverResponse.write(chunk);
-  if (!result) {
-    return new Promise(resolve => serverResponse.once('drain', resolve));
-  }
-}
+function safeWrite(
+  chunk: Buffer | Uint8Array | string,
+  serverResponse: NodeResponse,
+): Promise<void> | void {
+  // @ts-expect-error http and http2 writes are actually compatible
+  const result = serverResponse.write(chunk);
+  if (!result) {
+    return new Promise(resolve => {
+      const done = () => {
+        serverResponse.removeListener('error', done);
+        serverResponse.removeListener('close', done);
+        resolve();
+      };
+      serverResponse.once('drain', done);
+      serverResponse.once('error', done);
+      serverResponse.once('close', done);
+    });
+  }
+}

Benefits
• Guarantees promise resolution on 'drain', 'error', or 'close', preventing hangs.
• Narrows chunk’s type and adds an explicit return type for better type-safety.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
function safeWrite(chunk: any, serverResponse: NodeResponse) {
// @ts-expect-error http and http2 writes are actually compatible
const result = serverResponse.write(chunk);
if (!result) {
return new Promise(resolve => serverResponse.once('drain', resolve));
}
}
function safeWrite(
chunk: Buffer | Uint8Array | string,
serverResponse: NodeResponse,
): Promise<void> | void {
// @ts-expect-error http and http2 writes are actually compatible
const result = serverResponse.write(chunk);
if (!result) {
return new Promise(resolve => {
const done = () => {
serverResponse.removeListener('error', done);
serverResponse.removeListener('close', done);
resolve();
};
serverResponse.once('drain', done);
serverResponse.once('error', done);
serverResponse.once('close', done);
});
}
}
🤖 Prompt for AI Agents (early access)
In packages/server/src/utils.ts around lines 321 to 327, the safeWrite function currently only resolves its promise on the 'drain' event, which can cause it to hang if the socket closes or errors before 'drain' fires. To fix this, modify safeWrite to also listen for 'error' and 'close' events on serverResponse and resolve the promise when any of these events occur. Additionally, narrow the type of the chunk parameter and add an explicit return type to improve type safety.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
.changeset/whole-planes-mate.md (2)

5-6: Clarify hanging response description
The current wording is a bit wordy and contains a minor grammatical slip (“hangs” → “hang”), which could be streamlined for readability. Consider revising to something like:

- Some implementations like `compression` npm package do not implement `response.write(data, callback)` signature, but whatwg-node/server waits for it to finish the response stream.
- Then it causes the response stream hangs when the compression package takes the stream over when the response data is larger than its threshold.
+ Some middleware, such as the `compression` package, does not support the `response.write(data, callback)` signature. @whatwg-node/server currently waits for this callback before ending the response, which can cause the stream to hang on larger payloads.

8-10: Fix verb usage and framework naming
The phrase “workaround this” should be split into two words (“work around this”) when used as a verb. Also, use the official framework name “Express.js” when referring to the middleware. For example:

-It is actually a bug in `compression` package;
-[expressjs/compression#46](https://github.com/expressjs/compression/issues/46)
-But since it is a common mistake, we prefer to workaround this on our end.
+This is a bug in the Express.js `compression` middleware (see [expressjs/compression#46](https://github.com/expressjs/compression/issues/46)). Since this mistake is common, we prefer to work around it on our end.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~9-~9: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~10-~10: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 69830ee and 7c8d634.

📒 Files selected for processing (1)
  • .changeset/whole-planes-mate.md (1 hunks)
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[uncategorized] ~9-~9: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~10-~10: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (1)
.changeset/whole-planes-mate.md (1)

12-12: Changelog entry clearly summarizes the fix
The final sentence concisely describes the new behavior of calling response.end immediately after response.write. LGTM!

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
.changeset/whole-planes-mate.md (3)

5-6: Clarify and improve wording for readability
The current sentences are a bit convoluted; consider merging and refining to clearly explain the issue in one coherent statement.

- Some implementations like `compression` npm package do not implement `response.write(data, callback)` signature, but whatwg-node/server waits for it to finish the response stream.
- Then it causes the response stream hangs when the compression package takes the stream over when the response data is larger than its threshold.
+ Some middleware packages (e.g., `compression`) do not implement the `response.write(data, callback)` signature.  
+ The `@whatwg-node/server` adapter currently waits for the write callback to complete, which can cause the response stream to hang when the middleware intercepts larger payloads.

8-10: Fix punctuation and verb usage
Adjust punctuation for clarity, add the missing article before “compression” and correct the verb form “work around.”

- It is actually a bug in `compression` package;
- [expressjs/compression#46](https://github.com/expressjs/compression/issues/46)
- But since it is a common mistake, we prefer to workaround this on our end.
+ This is actually a bug in the `compression` package (expressjs/compression#46).  
+ However, since this mistake is common, we prefer to work around it on our end.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~9-~9: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~10-~10: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)


12-12: Enhance phrasing for consistency
Replace “So now” with “As a result” to improve flow and match formal changelog tone.

- So now the server adapter calls `response.end` immediately after `response.write` for static responses.
+ As a result, the server adapter now calls `response.end` immediately after `response.write` for static responses.
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3741d8a and e776335.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (5)
  • .changeset/whole-planes-mate.md (1 hunks)
  • packages/server/package.json (1 hunks)
  • packages/server/src/utils.ts (6 hunks)
  • packages/server/test/reproductions.spec.ts (2 hunks)
  • packages/server/test/test-server.ts (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • packages/server/test/test-server.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • packages/server/package.json
  • packages/server/test/reproductions.spec.ts
  • packages/server/src/utils.ts
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[uncategorized] ~9-~9: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~10-~10: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

⏰ Context from checks skipped due to timeout of 90000ms (20)
  • GitHub Check: e2e / cloudflare-modules
  • GitHub Check: e2e / cloudflare-workers
  • GitHub Check: e2e / azure-function
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / node 23
  • GitHub Check: unit / node 22
  • GitHub Check: unit / deno
  • GitHub Check: unit / node 20
  • GitHub Check: unit / bun
  • GitHub Check: alpha / snapshot
  • GitHub Check: unit / node 18
  • GitHub Check: server (undici)
  • GitHub Check: esm
  • GitHub Check: server (ponyfill)
  • GitHub Check: type check
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: lint
  • GitHub Check: server (native)
  • GitHub Check: prettier
  • GitHub Check: node-fetch (noConsumeBody)

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
.changeset/whole-planes-mate.md (2)

7-9: Fix grammatical error in hanging description.
The phrase "causes the response stream hangs" is grammatically incorrect and can be clearer. Consider rephrasing:

- Then it causes the response stream hangs when the compression package takes the stream over when the response data is larger than its threshold.
+ This can cause the response stream to hang when the compression middleware takes over for larger response payloads.

10-13: Correct framework naming and verb usage.

  • Change the semicolon after compression package to a period.
  • Capitalize the framework name as Express.js.
  • Use the verb form "work around" instead of the noun "workaround".
- It is actually a bug in `compression` package;
- [expressjs/compression#46](https://github.com/expressjs/compression/issues/46)
- But since it is a common mistake, we prefer to workaround this on our end.
+ It is actually a bug in the `compression` package.
+ [Express.js/compression#46](https://github.com/expressjs/compression/issues/46)
+ But since it is a common mistake, we prefer to work around this on our end.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~11-~11: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~12-~12: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

packages/node-fetch/src/FormData.ts (1)

132-155: Node-specific Buffer usage ties the ponyfill to Node and adds unnecessary conversions

All header/body fragments are wrapped in Buffer.from(...), but the public contract (PonyfillReadableStream<Uint8Array>) only promises Uint8Array. While Buffer is a subclass, relying on it:

  1. Couples the ponyfill to Node (e.g. Deno or Cloudflare Workers won’t have Buffer).
  2. Performs an extra allocation (Buffer.from(string) does a copy) where TextEncoder would be sufficient.

A minimal, environment-agnostic change would be:

- controller.enqueue(Buffer.from(`--${boundary}\r\n`));
+ controller.enqueue(new TextEncoder().encode(`--${boundary}\r\n`));

The same applies to every other controller.enqueue(Buffer.from(...)) call.

This keeps compatibility while reducing Node-only surface area and avoids the implicit copy that Buffer.from incurs.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e776335 and fcbe4f1.

📒 Files selected for processing (2)
  • .changeset/whole-planes-mate.md (1 hunks)
  • packages/node-fetch/src/FormData.ts (1 hunks)
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[uncategorized] ~11-~11: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~12-~12: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. So now the server ada...

(NOUN_VERB_CONFUSION)

⏰ Context from checks skipped due to timeout of 90000ms (19)
  • GitHub Check: e2e / cloudflare-modules
  • GitHub Check: e2e / cloudflare-workers
  • GitHub Check: e2e / azure-function
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / deno
  • GitHub Check: unit / node 23
  • GitHub Check: esm
  • GitHub Check: unit / bun
  • GitHub Check: unit / node 22
  • GitHub Check: unit / node 20
  • GitHub Check: server (undici)
  • GitHub Check: unit / node 18
  • GitHub Check: type check
  • GitHub Check: server (ponyfill)
  • GitHub Check: server (native)
  • GitHub Check: prettier
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: lint
  • GitHub Check: node-fetch (noConsumeBody)
🔇 Additional comments (2)
.changeset/whole-planes-mate.md (2)

1-5: Changelog frontmatter format is correct.


14-15: Changelog description of workaround is clear.
The summary of the workaround succinctly describes the change to call response.end immediately after response.write for static responses. No issues found.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
packages/node-fetch/src/FormData.ts (1)

99-99: ⚠️ Potential issue

Incorrect type for currentAsyncIterator will break type-safety

The currentAsyncIterator is declared with type AsyncIterator<[string, FormDataEntryValue]>, but on line 127 it's assigned entryStream[Symbol.asyncIterator]() which yields Uint8Array or Buffer chunks, not key-value pairs. This type mismatch will cause issues when using the iterator's values.

-  let currentAsyncIterator: AsyncIterator<[string, FormDataEntryValue]> | undefined;
+  let currentAsyncIterator: AsyncIterator<Uint8Array | Buffer> | undefined;
🧹 Nitpick comments (5)
.changeset/whole-planes-mate.md (2)

10-12: Grammar and style improvements

  • Capitalize the framework name: use Express.js instead of expressjs.
  • Use the verb phrase work around instead of the noun workaround.
  • Replace the semicolon after “package” with a period for better readability.

Example suggestion:

-It is actually a bug in `compression` package; [expressjs/compression#46](...)
+It is actually a bug in the `compression` package. [Express.js/compression#46](...)

-But since it is a common mistake, we prefer to workaround this on our end.
+But since it is a common mistake, we prefer to work around this on our end.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~11-~11: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~12-~12: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. Now after calling `re...

(NOUN_VERB_CONFUSION)


14-18: Refine compound sentences and punctuation

  • Add a comma before “but” when linking independent clauses.
  • Replace the semicolon with a period or comma to improve flow.
  • Insert a comma before “and” in the compound sentence for consistency.

Suggested revision:

-Now after calling `response.write`, it no longer uses callback but first it checks the result;
+Now, after calling `response.write`, it no longer uses a callback, but first it checks the result.

-if it is `true`, it means stream is drained and we can call `response.end` immediately.
+If the result is `true`, the stream is drained, and we can call `response.end` immediately.

-else if it is `false`, it means the stream is not drained yet, so we can wait for the `drain` event to call `response.end`.
+Otherwise, if it is `false`, the stream is not yet drained, so we wait for the `drain` event before calling `response.end`.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~14-~14: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ponse.write, it no longer uses callback but first it checks the result; if it is ...

(COMMA_COMPOUND_SENTENCE)


[uncategorized] ~16-~16: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...it is true, it means stream is drained and we can call response.end immediately....

(COMMA_COMPOUND_SENTENCE)

packages/node-fetch/src/FormData.ts (3)

132-158: Improve performance with single pass processing

The current implementation alternates between processing form boundaries and streaming file contents, which can be inefficient for large forms with many files. Consider implementing a queue-based approach that batches headers and boundaries.

The current implementation is functionally correct, but for forms with many file entries, the constant switching between iterators might introduce overhead. A more efficient approach could buffer headers and process entries more contiguously.


155-158: Enhance error handling in cancel method

The cancel method should handle errors that might occur when calling return on iterators.

    cancel: err => {
-      entriesIterator?.return?.(err);
-      currentAsyncIterator?.return?.(err);
+      try {
+        entriesIterator?.return?.(err);
+      } catch (error) {
+        console.error('Error closing entriesIterator:', error);
+      }
+      try {
+        currentAsyncIterator?.return?.(err);
+      } catch (error) {
+        console.error('Error closing currentAsyncIterator:', error);
+      }
    },

93-96: Consider using a more unique boundary string

The default boundary string '---' is very simple and might appear in the content. Consider using a more unique default.

export function getStreamFromFormData(
  formData: FormData,
-  boundary = '---',
+  boundary = `----FormBoundary${Math.random().toString(36).substring(2)}`,
): PonyfillReadableStream<Uint8Array> {
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2064db7 and b5d5ecf.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (10)
  • .changeset/whole-planes-mate.md (1 hunks)
  • packages/node-fetch/src/FormData.ts (1 hunks)
  • packages/node-fetch/src/TransformStream.ts (2 hunks)
  • packages/node-fetch/src/WritableStream.ts (3 hunks)
  • packages/node-fetch/src/fetchNodeHttp.ts (4 hunks)
  • packages/node-fetch/src/utils.ts (1 hunks)
  • packages/server/package.json (1 hunks)
  • packages/server/src/utils.ts (6 hunks)
  • packages/server/test/reproductions.spec.ts (2 hunks)
  • packages/server/test/test-server.ts (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (8)
  • packages/server/package.json
  • packages/node-fetch/src/TransformStream.ts
  • packages/node-fetch/src/utils.ts
  • packages/node-fetch/src/fetchNodeHttp.ts
  • packages/server/test/test-server.ts
  • packages/node-fetch/src/WritableStream.ts
  • packages/server/src/utils.ts
  • packages/server/test/reproductions.spec.ts
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[uncategorized] ~11-~11: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~12-~12: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. Now after calling `re...

(NOUN_VERB_CONFUSION)


[uncategorized] ~14-~14: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ponse.write, it no longer uses callback but first it checks the result; if it is ...

(COMMA_COMPOUND_SENTENCE)


[uncategorized] ~16-~16: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...it is true, it means stream is drained and we can call response.end immediately....

(COMMA_COMPOUND_SENTENCE)

⏰ Context from checks skipped due to timeout of 90000ms (20)
  • GitHub Check: e2e / cloudflare-modules
  • GitHub Check: e2e / cloudflare-workers
  • GitHub Check: e2e / azure-function
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / node 23
  • GitHub Check: unit / deno
  • GitHub Check: unit / node 22
  • GitHub Check: unit / node 20
  • GitHub Check: unit / bun
  • GitHub Check: server (undici)
  • GitHub Check: unit / node 18
  • GitHub Check: server (ponyfill)
  • GitHub Check: esm
  • GitHub Check: alpha / snapshot
  • GitHub Check: prettier
  • GitHub Check: node-fetch (consumeBody)
  • GitHub Check: type check
  • GitHub Check: server (native)
  • GitHub Check: node-fetch (noConsumeBody)
  • GitHub Check: lint
🔇 Additional comments (2)
.changeset/whole-planes-mate.md (2)

1-5: Verify changeset frontmatter format

The YAML frontmatter correctly lists each package with the patch release type, matching the repository’s changeset conventions. No changes needed here.


7-9: Clarify description of hanging stream

The explanation clearly describes why the response stream can hang when using compression with larger payloads. It concisely sets up the motivation for the workaround.

Comment on lines +146 to +149
if (value) {
return controller.enqueue(value);
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add type checking for stream values

After fixing the currentAsyncIterator type, we should ensure that the values being enqueued are of the expected type.

           if (value) {
-            return controller.enqueue(value);
+            return controller.enqueue(Buffer.isBuffer(value) ? value : Buffer.from(value));
           } else {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (value) {
return controller.enqueue(value);
} else {
if (value) {
return controller.enqueue(Buffer.isBuffer(value)
? value
: Buffer.from(value));
} else {
🤖 Prompt for AI Agents (early access)
In packages/node-fetch/src/FormData.ts around lines 146 to 148, the code
enqueues a value without verifying its type. To fix this, add a type check
before enqueuing to ensure the value matches the expected stream type. This
prevents runtime errors by only enqueuing valid stream values.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
packages/node-fetch/src/FormData.ts (1)

147-148: Add type checking for stream values

When enqueueing values from the async iterator, there's no type verification which could lead to runtime errors if unexpected data comes through the stream.

          if (value) {
-            return controller.enqueue(value);
+            return controller.enqueue(Buffer.isBuffer(value) ? value : Buffer.from(value));
          } else {
🧹 Nitpick comments (1)
packages/node-fetch/src/FormData.ts (1)

127-128: Remove @ts-expect-error by using proper typing

There's a type error being suppressed with @ts-expect-error. Consider fixing the underlying type issue rather than suppressing it.

-        const entryStream = blobOrString.stream();
-        // @ts-expect-error - ReadableStream is async iterable
-        currentAsyncIterator = entryStream[Symbol.asyncIterator]();
+        const entryStream: ReadableStream<Uint8Array> = blobOrString.stream();
+        currentAsyncIterator = entryStream[Symbol.asyncIterator]();

Alternatively, you could create a type guard or assertion function to safely access the asyncIterator.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f753682 and 9b4d92c.

📒 Files selected for processing (5)
  • packages/node-fetch/src/FormData.ts (1 hunks)
  • packages/promise-helpers/src/index.ts (1 hunks)
  • packages/server/src/createServerAdapter.ts (1 hunks)
  • packages/server/src/utils.ts (7 hunks)
  • test.mjs (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • test.mjs
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/server/src/utils.ts
🧰 Additional context used
🧬 Code Graph Analysis (1)
packages/promise-helpers/src/index.ts (3)
packages/node-fetch/src/utils.ts (1)
  • fakePromise (30-30)
packages/server/src/uwebsockets.ts (1)
  • fakePromise (277-277)
packages/server/src/utils.ts (1)
  • isPromise (14-14)
🔇 Additional comments (8)
packages/promise-helpers/src/index.ts (3)

35-42: Optimization for the case with finallyFactory looks good.

The code now properly distinguishes between cases where a finallyFactory is provided, retaining the original implementation for those cases while enabling a more optimized path for cases without the finally handler.


43-56: Performance optimization for promise handling.

This implementation optimizes handling of different promise types by:

  1. Directly extracting values from fake promises without chaining
  2. Properly handling fake rejected promises
  3. Only creating promise chains when actually needed
  4. Using synchronous execution paths when possible

This should reduce unnecessary promise creation and improve performance in the common case.


57-64: Good error handling strategy.

The implementation correctly handles errors differently based on whether an error factory is provided:

  • When no error factory exists, it allows errors to propagate naturally
  • When an error factory exists, it catches synchronous errors and redirects to the error factory

This maintains consistent behavior with the original implementation while optimizing performance.

packages/node-fetch/src/FormData.ts (4)

97-99: Type issue fixed for currentAsyncIterator

The type definition for currentAsyncIterator has been correctly updated to AsyncIterator<Uint8Array> | undefined which properly reflects the actual data type yielded by blob streams. This resolves the type safety concerns mentioned in previous reviews.


156-159: Good addition of cancel method for proper resource cleanup

The addition of the cancel method with proper cleanup of both iterators is an excellent practice for resource management in stream implementations. The use of optional chaining (?.) helps avoid errors if the methods don't exist.


101-132: Well-structured handleNextEntry function

This function effectively encapsulates the logic for processing FormData entries, making the code more maintainable. It correctly handles both string and blob values, generating appropriate multipart form data boundaries and headers.


133-155: Improved stream handling with async processing

The refactored implementation properly handles asynchronous streaming of form data entries with better backpressure management. The conditional logic in the pull method ensures entries are processed one at a time, moving to the next entry only after the current one is fully processed.

packages/server/src/createServerAdapter.ts (1)

280-286:

✅ Verification successful

Implementation improved to handle middleware that doesn't properly implement response.write callbacks.

The refactored approach correctly passes the nodeResponse directly to normalizeNodeRequest instead of using a weakmap. This aligns with the PR objective to fix hanging responses when using middleware like the compression package that doesn't properly implement the response.write(data, callback) signature.

Let's verify how the new implementation handles Node.js response streams:


🏁 Script executed:

#!/bin/bash
# Description: Check the implementation details of normalizeNodeRequest function
# to understand how it now handles the nodeResponse parameter

rg -A 10 "normalizeNodeRequest" --glob "packages/server/src/utils.ts"

Length of output: 802


🏁 Script executed:

#!/bin/bash
# Inspect the full implementation of normalizeNodeRequest
echo "---- normalizeNodeRequest (with context) ----"
rg -n "export function normalizeNodeRequest" -A 200 --glob packages/server/src/utils.ts

echo
echo "---- All references to nodeResponse in utils.ts ----"
rg -n "nodeResponse" --glob packages/server/src/utils.ts

echo
echo "---- Any WeakMap usage in the repo ----"
rg -i -n "weakmap" -C 3

Length of output: 17558


🏁 Script executed:

#!/bin/bash
# Check for any WeakMap usage in createServerAdapter.ts
echo "---- WeakMap usage in createServerAdapter.ts ----"
rg -n "WeakMap" -C 5 --glob packages/server/src/createServerAdapter.ts || echo "No WeakMap usages found."

echo
# Locate handleRequest invocation(s) to see how nodeResponse flows
echo "---- handleRequest calls in createServerAdapter.ts ----"
rg -n "handleRequest" -C 5 --glob packages/server/src/createServerAdapter.ts || echo "No handleRequest calls found."

echo
# Check for decompressedResponseMap references in this file
echo "---- decompressedResponseMap references ----"
rg -n "decompressedResponseMap" -C 5 --glob packages/server/src/createServerAdapter.ts || echo "No decompressedResponseMap references found."

Length of output: 10964


Pass nodeResponse into normalizeNodeRequest to bind abort signals

  • File: packages/server/src/createServerAdapter.ts
  • Lines: 281–286
     // Ensure `waitUntil` is available in the server context
     if (!serverContext.waitUntil) {
       serverContext.waitUntil = waitUntil;
     }
-    const request = normalizeNodeRequest(nodeRequest, fetchAPI);
+    const request = normalizeNodeRequest(nodeRequest, fetchAPI, nodeResponse);
     return handleRequest(request, serverContext);

This change forwards the Node.js nodeResponse directly into normalizeNodeRequest, which now registers error, close, and finish listeners on the response stream to abort the request body’s AbortController when the response ends unexpectedly. It prevents hanging requests when middleware (e.g., compression) doesn’t invoke the response.write(data, callback) callback correctly.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
packages/node-fetch/src/FormData.ts (1)

147-149: Add type checking for stream values

There's no validation that value is of the expected type before enqueueing. This was noted in a previous review comment but hasn't been addressed.

           if (value) {
-            return controller.enqueue(value);
+            return controller.enqueue(Buffer.isBuffer(value) ? value : Buffer.from(value));
           } else {
🧹 Nitpick comments (4)
packages/node-fetch/src/FormData.ts (2)

126-128: Consider removing the TypeScript error suppression comment

The @ts-expect-error comment suggests there's a typing issue with the ReadableStream's async iterable interface. Consider addressing this more directly with appropriate type declarations or interfaces.

-        // @ts-expect-error - ReadableStream is async iterable
-        currentAsyncIterator = entryStream[Symbol.asyncIterator]();
+        // ReadableStream's AsyncIterator yields Uint8Array chunks
+        currentAsyncIterator = entryStream[Symbol.asyncIterator]() as AsyncIterator<Uint8Array>;

133-155: Consider adding explicit error handling

The implementation lacks explicit error handling in the pull callback. Consider adding try/catch blocks or error handling for the promises to prevent stream crashes due to errors in blob processing.

    pull: controller => {
      if (!sentInitialHeader) {
        sentInitialHeader = true;
        return controller.enqueue(Buffer.from(`--${boundary}\r\n`));
      }
      if (currentAsyncIterator) {
-        return currentAsyncIterator.next().then(({ done, value }) => {
-          if (done) {
-            currentAsyncIterator = undefined;
-          }
-          if (value) {
-            return controller.enqueue(value);
-          } else {
-            return handleNextEntry(controller);
-          }
-        });
+        return currentAsyncIterator.next()
+          .then(({ done, value }) => {
+            if (done) {
+              currentAsyncIterator = undefined;
+            }
+            if (value) {
+              return controller.enqueue(Buffer.isBuffer(value) ? value : Buffer.from(value));
+            } else {
+              return handleNextEntry(controller);
+            }
+          })
+          .catch(error => {
+            // Log the error or transform it to a more user-friendly format
+            console.error('Error processing form data stream:', error);
+            controller.error(error);
+          });
      }
      return handleNextEntry(controller);
    },
.changeset/whole-planes-mate.md (2)

7-12: Improve clarity and correct grammar in description.

Lines 7–12 contain some grammatical issues and can be made clearer. Suggested diff:

- Some implementations like `compression` npm package do not implement `response.write(data, callback)` signature, but whatwg-node/server waits for it to finish the response stream.
- Then it causes the response stream hangs when the compression package takes the stream over when the response data is larger than its threshold.
- It is actually a bug in `compression` package; [expressjs/compression#46](https://github.com/expressjs/compression/issues/46)
- But since it is a common mistake, we prefer to workaround this on our end.
+ Some implementations, like the `compression` npm package, do not implement the `response.write(data, callback)` signature, but `@whatwg-node/server` currently waits for the write callback to finish before ending the response stream.
+ This causes the response to hang when the compression middleware takes over for larger payloads.
+ This is a bug in the `compression` package (see [Express.js compression issue #46](https://github.com/expressjs/compression/issues/46)).
+ Since this mistake is common in other middleware, we prefer to work around it on our end.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~11-~11: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~12-~12: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. Now after calling `re...

(NOUN_VERB_CONFUSION)


14-17: Refine workaround description for readability and accuracy.

Lines 14–17 can be rephrased for better flow and correct punctuation. Suggested diff:

- Now after calling `response.write`, it no longer uses callback but first it checks the result;
- if it is `true`, it means stream is drained and we can call `response.end` immediately.
- else if it is `false`, it means the stream is not drained yet, so we can wait for the `drain` event to call `response.end`.
+ After calling `response.write`, we no longer rely on its callback; instead, we check its return value:
+ - If it returns `true`, the stream is drained, and we call `response.end` immediately.
+ - If it returns `false`, the stream is not yet drained, so we wait for the `drain` event before calling `response.end`.
🧰 Tools
🪛 LanguageTool

[uncategorized] ~14-~14: You might be missing the article “a” here.
Context: ...ing response.write, it no longer uses callback but first it checks the result; if it ...

(AI_EN_LECTOR_MISSING_DETERMINER_A)


[uncategorized] ~14-~14: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ponse.write, it no longer uses callback but first it checks the result; if it is ...

(COMMA_COMPOUND_SENTENCE)


[uncategorized] ~16-~16: You might be missing the article “the” here.
Context: ... the result; if it is true, it means stream is drained and we can call `response.en...

(AI_EN_LECTOR_MISSING_DETERMINER_THE)


[uncategorized] ~16-~16: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...it is true, it means stream is drained and we can call response.end immediately....

(COMMA_COMPOUND_SENTENCE)


[uncategorized] ~16-~16: Possible missing comma found.
Context: ...we can call response.end immediately. else if it is false, it means the stream i...

(AI_HYDRA_LEO_MISSING_COMMA)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9b4d92c and 25d5aec.

⛔ Files ignored due to path filters (1)
  • yarn.lock is excluded by !**/yarn.lock, !**/*.lock
📒 Files selected for processing (13)
  • .changeset/whole-planes-mate.md (1 hunks)
  • packages/node-fetch/src/FormData.ts (1 hunks)
  • packages/node-fetch/src/TransformStream.ts (2 hunks)
  • packages/node-fetch/src/WritableStream.ts (3 hunks)
  • packages/node-fetch/src/fetchNodeHttp.ts (4 hunks)
  • packages/node-fetch/src/utils.ts (1 hunks)
  • packages/promise-helpers/src/index.ts (1 hunks)
  • packages/server/package.json (1 hunks)
  • packages/server/src/createServerAdapter.ts (1 hunks)
  • packages/server/src/utils.ts (7 hunks)
  • packages/server/test/reproductions.spec.ts (2 hunks)
  • packages/server/test/test-server.ts (1 hunks)
  • test.mjs (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (11)
  • packages/node-fetch/src/TransformStream.ts
  • test.mjs
  • packages/node-fetch/src/fetchNodeHttp.ts
  • packages/promise-helpers/src/index.ts
  • packages/node-fetch/src/utils.ts
  • packages/server/package.json
  • packages/server/test/reproductions.spec.ts
  • packages/server/src/createServerAdapter.ts
  • packages/node-fetch/src/WritableStream.ts
  • packages/server/test/test-server.ts
  • packages/server/src/utils.ts
🧰 Additional context used
🪛 LanguageTool
.changeset/whole-planes-mate.md

[uncategorized] ~11-~11: The official spelling of this programming framework is “Express.js”.
Context: ...tually a bug in compression package; [expressjs/compression#46](https://github.com/expr...

(NODE_JS)


[grammar] ~12-~12: The word “workaround” is a noun. The verb is spelled with a white space.
Context: ...ce it is a common mistake, we prefer to workaround this on our end. Now after calling `re...

(NOUN_VERB_CONFUSION)


[uncategorized] ~14-~14: You might be missing the article “a” here.
Context: ...ing response.write, it no longer uses callback but first it checks the result; if it ...

(AI_EN_LECTOR_MISSING_DETERMINER_A)


[uncategorized] ~14-~14: Use a comma before ‘but’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...ponse.write, it no longer uses callback but first it checks the result; if it is ...

(COMMA_COMPOUND_SENTENCE)


[uncategorized] ~16-~16: You might be missing the article “the” here.
Context: ... the result; if it is true, it means stream is drained and we can call `response.en...

(AI_EN_LECTOR_MISSING_DETERMINER_THE)


[uncategorized] ~16-~16: Use a comma before ‘and’ if it connects two independent clauses (unless they are closely connected and short).
Context: ...it is true, it means stream is drained and we can call response.end immediately....

(COMMA_COMPOUND_SENTENCE)


[uncategorized] ~16-~16: Possible missing comma found.
Context: ...we can call response.end immediately. else if it is false, it means the stream i...

(AI_HYDRA_LEO_MISSING_COMMA)

⏰ Context from checks skipped due to timeout of 90000ms (8)
  • GitHub Check: e2e / azure-function
  • GitHub Check: unit / node 22
  • GitHub Check: unit / node 20
  • GitHub Check: e2e / aws-lambda
  • GitHub Check: unit / node 23
  • GitHub Check: unit / node 18
  • GitHub Check: unit / deno
  • GitHub Check: unit / bun
🔇 Additional comments (4)
packages/node-fetch/src/FormData.ts (3)

97-100: Type correction for currentAsyncIterator improves type safety

The type has been correctly updated to AsyncIterator<Uint8Array> to match the actual data yielded by blob streams, fixing the type mismatch that was previously flagged in a review comment.


101-132: Implementation of incremental streaming with entry-by-entry processing

The new handleNextEntry function implements a more efficient streaming approach that processes form data entries on demand rather than buffering them all upfront. The function properly handles boundaries between entries and correctly differentiates between string and Blob entries.


156-159: Good addition of cancellation handling

The new cancellation handler properly cleans up both iterators when the stream is cancelled, which is essential for proper resource management in streaming operations.

.changeset/whole-planes-mate.md (1)

1-5: Metadata frontmatter is correct.

The changeset header and package bump declarations are properly formatted and conform to the expected YAML frontmatter syntax.

@ardatan ardatan merged commit 9527e8f into master May 9, 2025
25 of 26 checks passed
@ardatan ardatan deleted the fix-write-end branch May 12, 2025 16:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

upgrading @whatwg-node/server:0.10.3+ breaks express compression middleware
2 participants