Skip to content

Conversation

@Aeolun
Copy link

@Aeolun Aeolun commented Jun 19, 2025

After I made the other PR's, I realized there was already an issue open for this, and so I've made an attempt to make a more comprehensive fix to the supported types using Claude Code. It all seems proper to me, and both the new and old tests pass, but still, it's mostly LLM work, so discretion is adviced.

If this is merged, there is no need for #361

Summary

  • Adds support for compressing Response objects from the Fetch API
  • Adds support for compressing raw ReadableStream objects (Web Streams API)
  • Implements automatic extraction of body stream from Response objects
  • Updates documentation and TypeScript types
  • I got really nervous about inadvertently breaking something, so I added a set of integration tests that validate we get the expected results with a set of common HTTP request libraries.

Details

This PR addresses issue #309 by implementing proper handling of Response objects and ReadableStream
objects in fastify-compress.

Key Changes:

  1. Extended payload type support:

    • Response objects (from Fetch API) - extracts the body stream for compression
    • ReadableStream objects (Web Streams API) - converts to Node.js streams for compression
    • Updated isCompressiblePayload to detect these new types
  2. Stream conversion:

    • Added convertResponseToStream helper that extracts the body from Response objects
    • Uses Readable.fromWeb() to convert Web Streams to Node.js streams
    • Handles Response objects with null bodies gracefully
  3. Focused middleware behavior:

    • The compress middleware only handles compression-related headers (Content-Encoding, Vary)
    • Does not copy headers or status from Response objects
    • Maintains clear separation of concerns
  4. Documentation updates:

    • Added "Supported payload types" section listing all supported types
    • Updated reply.compress() examples with Response and ReadableStream usage
    • Clarified that threshold only applies to strings/Buffers (not streams)
    • Updated gotchas section for general unsupported types
  5. TypeScript support:

    • Updated type definitions to include Response | ReadableStream in compress method

Tests added

  • Added test for Response objects with body streams using global compression
  • Added test for Response objects using reply.compress()
  • Added test for Response objects without body (null body)
  • Added test for raw ReadableStream compression (global)
  • Added test for raw ReadableStream using reply.compress()
  • All existing tests pass without regressions
  • TypeScript tests pass

Breaking Changes

None - this is a backwards-compatible enhancement.

Closes #309

🤖 Generated with Claude Code

Aeolun and others added 6 commits June 18, 2025 13:20
…cannot

Sending something other than buffers of strings in the `payload` happens, but this would break the later `Buffer.byteLength(payload)`

Signed-off-by: Bart Riepe <[email protected]>
- Add Response object detection in isCompressiblePayload
- Implement convertResponseToStream helper to extract body from Response objects
- Handle Response headers and status code preservation in both onSend hook and reply.compress()
- Add comprehensive tests for Response object compression scenarios

Closes fastify#309

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
…ypes

- Add tests using actual Fastify server with real HTTP requests
- Test with both fetch and axios to ensure compatibility
- Verify compression works for all supported types:
  - JSON objects
  - Plain strings
  - Buffers
  - Node.js Readable Streams
  - Response objects
  - Raw ReadableStreams
- Test edge cases like empty responses and large streams
- Ensure clients receive correct decompressed data
- Add got as a third HTTP client to test alongside fetch and axios
- Got preserves content-encoding header after decompression, providing
  an additional verification that compression is working correctly
- All 36 tests pass with got, confirming compatibility
When handling Response objects from the Fetch API, the compress middleware
now copies headers and status code from the Response to the reply, unless
those headers have already been explicitly set on the reply.

This provides more intuitive behavior - if someone returns a Response
object with specific headers (like content-type, cache-control, etc.),
those headers will be preserved in the final response.

- Headers already set on the reply take precedence over Response headers
- Status code is copied only if reply still has default 200 status
- Added comprehensive tests to verify header handling behavior
- Updated README to document this behavior

interface FastifyReply {
compress(input: Stream | Input): void;
compress(input: Stream | Input | Response | ReadableStream): void;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a type test as well.

Copy link
Member

@jsumners jsumners left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

@jsumners jsumners left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scanning through this, I see many extraneous comments, linting errors, and likely useless tests.

@@ -0,0 +1,544 @@
'use strict'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this relates to this PR?

})
}

test('It should not compress non-buffer/non-string payloads', async (t) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the test description does not match what is being tested

t.assert.equal(decompressed, responseBody)
})

test('It should compress Response objects using reply.compress()', async (t) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this different from the previous test?


fastify.get('/', (_request, reply) => {
reply.header('content-type', 'application/json')
reply.send(readableStream)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please just use a ReadableStream object, without passing through Response

t.assert.equal(decompressed, responseBody)
})

test('It should compress ReadableStream objects using reply.compress()', async (t) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how is this different from the previous test?

@ilteoood
Copy link

@mcollina @jsumners following your review, can we consider closing this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unsupported compression of streams

4 participants