Skip to content

Add gzip compression using Compression Streams API #481

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
May 7, 2025

Conversation

raikasdev
Copy link

Problem

Adds GZIP compression support for json capturemode

Closes #254

Changes

Simple wrapper around CompressionStream and a function to check if the class is available

Release info Sub-libraries affected

Bump level

  • Major
  • Minor
  • Patch

Libraries affected

  • All of them
  • posthog-web
  • posthog-node
  • posthog-ai
  • posthog-react-native

Changelog notes

  • Added support for GZIP compression in posthog-js-lite using Web APIs

@raikasdev raikasdev requested a review from marandaneto as a code owner April 29, 2025 14:53
@raikasdev raikasdev mentioned this pull request Apr 29, 2025
4 tasks
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

Added GZIP compression support for JSON capture mode using the Compression Streams API, improving data transmission efficiency for large payloads.

  • Added gzip.ts with feature detection and compression functionality using CompressionStream
  • Modified batch endpoint URL to include ?compression=gzip-js parameter when GZIP is enabled
  • Updated PostHogFetchOptions type to support Blob body for compressed data
  • Added runtime check via isGzipSupported() to gracefully handle environments without compression support
  • Set Content-Type header to text/plain for GZIP compressed requests

💡 (1/5) You can manually trigger the bot by mentioning @greptileai in a comment!

3 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings | Greptile

@marandaneto
Copy link
Member

@raikasdev the remote config/decide API will return supportedCompression, we need to check if its acceptable on the server side as well.
you can check the PostHogDecideResponse object.
and how this is used on the JS SDK: https://github.com/PostHog/posthog-js/blob/db26fffea82197b74e5f1f3c0b240b00d0e363e8/src/types.ts#L1255-L1258

@raikasdev raikasdev requested a review from marandaneto April 29, 2025 15:27
@marandaneto
Copy link
Member

@raikasdev once you fix tests (maybe you have to forcefully disable compression in tests)/lint I can test it out.
some unit tests would be awesome as well regarding the supported compressions and cangzip

@raikasdev
Copy link
Author

Fixed tests requiring JSON bodies and added few tests for gzip compress using Node's builtin gzip (via node:zlib)

@marandaneto
Copy link
Member

body: `data=${encodeURIComponent(LZString.compressToBase64(payload))}&compression=lz64`,

on this line we should also check for disableCompression and supportedCompression

@raikasdev
Copy link
Author

Test should be fixed, weird that it worked locally without the import 😄. I'll look at the comments later today

@raikasdev
Copy link
Author

@marandaneto If lz64 is getting deprecated should the captureMode form also be using gzip if it is available in the runtime?

@marandaneto
Copy link
Member

@marandaneto If lz64 is getting deprecated should the captureMode form also be using gzip if it is available in the runtime?

this will be addressed later on with another issue/PR #483
lets keep this one simple for now.

@raikasdev raikasdev requested a review from marandaneto May 2, 2025 11:42
@marandaneto
Copy link
Member

@raikasdev thanks
one last comment and fix the lint and GG

@raikasdev
Copy link
Author

Weird, locally prettier doesn't show anything 🤔

@marandaneto
Copy link
Member

Weird, locally prettier doesn't show anything 🤔

probably versioning, lets see if this helps 272e099 (#481)

@marandaneto marandaneto merged commit 6a4bffb into PostHog:v5 May 7, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants