Skip to content

feat: JWT cache implementation based on sieve algorithm - #4084

Merged
steve-chavez merged 3 commits into
PostgREST:mainfrom
mkleczek:jwt-cache-sieve
Jul 29, 2025
Merged

feat: JWT cache implementation based on sieve algorithm#4084
steve-chavez merged 3 commits into
PostgREST:mainfrom
mkleczek:jwt-cache-sieve

Conversation

@mkleczek

Copy link
Copy Markdown
Collaborator

Draft version of JWT cache implementation based on https://cachemon.github.io/SIEVE-website/blog/2023/12/17/sieve-is-simpler-than-lru/ algorithm

@mkleczek

mkleczek commented May 14, 2025

Copy link
Copy Markdown
Collaborator Author

This PR contains:

  1. Refactoring and some cleanup of JWT handling code:
    • Moved JWT parsing and validation to a separate module Auth.JWT
    • Split JWT decoding, parsing and signature validation, claims validation into separate functions
    • Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
    • Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
    • handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module
  2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also provides usage stats (ie. hit ratio, evictions count, size)

@steve-chavez

Copy link
Copy Markdown
Member

Very interesting! 👀 👀

It also provides usage stats (ie. hit ratio, evictions count, size)

Maybe we could expose those as metrics. That would help with passing code coverage too, since it looks like it's detecting those functions as dead code:

postgrest-13.1-inplace: src/PostgREST/Cache/Sieve.hs:124:1: delete
postgrest-13.1-inplace: src/PostgREST/Cache/Sieve.hs:133:1: deleteIO
postgrest-13.1-inplace: src/PostgREST/Cache/Sieve.hs:136:1: resetIO
postgrest-13.1-inplace: src/PostgREST/Cache/Sieve.hs:142:1: accessStats
postgrest-13.1-inplace: src/PostgREST/Cache/Sieve.hs:148:1: evictionsCount

I also see the JWT loadtest failing on CI https://github.com/PostgREST/postgrest/actions/runs/15030111955?pr=4084:

Options "http://postgrest/authors_only": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Not sure what's going on there, but you should be able to reproduce it locally with postgrest-loadtest -k jwt.

@mkleczek

mkleczek commented May 15, 2025

Copy link
Copy Markdown
Collaborator Author

postgrest-loadtest -k jwt gives me:

delaying data to/from PostgREST by 0ms
jwt": -c: line 1: unexpected EOF while looking for matching `"'

Not sure what the problem is - tried to find typos in my changes to env variables settings, main (0b3c8c9) has the same issue on my machine.

@steve-chavez

Copy link
Copy Markdown
Member

postgrest-loadtest -k jwt gives me:
delaying data to/from PostgREST by 0ms
jwt": -c: line 1: unexpected EOF while looking for matching `"'

Weird, it's like you're using and old version of the nix tools and it's ignoring the -k jwt arg. On my machine I get:

$ postgrest-loadtest -k jwt
Created 50000 targets in ./test/load/gen_targets.http (0.79s)
...

## this is the ouput you're getting
$ postgrest-loadtest -k mixed # same as just "postgrest-loadtest"
delaying data to/from postgres by 0ms 

Maybe try going out of nix-shell and running it again.

@mkleczek

Copy link
Copy Markdown
Collaborator Author

Yeah, that was it, thanks.

I have serious doubts about postgrest-loadtest -k jwt. When testing main on my machine (beefy MacBook) there is no difference in results between PGRST_JWT_CACHE_MAX_LIFETIME=86400 and PGRST_JWT_CACHE_MAX_LIFETIME=0. I guess that's because symmetric crypto is used to generate/validate tokens. That's probably too fast to be worth caching and show any difference between cached and not cached access. I think meaningful tests have to be based on asymmetric cryptography.

Nevertheless there indeed seems to be an issue with implementation in this PR: all percentiles (ie. median up to 99th) show better results than main but max is 30s (which causes timeouts and skews average response time). Looking at it right now.

@steve-chavez

Copy link
Copy Markdown
Member

That's probably too fast to be worth caching and show any difference between cached and not cached access. I think meaningful tests have to be based on asymmetric cryptography.

Agree, we can change that.

When testing main on my machine (beefy MacBook) there is no difference in results between PGRST_JWT_CACHE_MAX_LIFETIME=86400 and PGRST_JWT_CACHE_MAX_LIFETIME=0.

Note that the jwt loadtest actually generates unique JWTs , it was mainly done to test the jwt decoding perf while also ensuring the JWT cache purging doesn't slow things down (#4034).

The "mixed" loadtest does have hardcoded JWTs, and the perf looks more or less maintained (or slightly better).

@mkleczek
mkleczek marked this pull request as ready for review May 16, 2025 09:09
@mkleczek mkleczek changed the title feat: (Draft) JWT cache implementation based on sieve algorithm feat: JWT cache implementation based on sieve algorithm May 16, 2025
@mkleczek

Copy link
Copy Markdown
Collaborator Author

@steve-chavez @wolfgangwalther @taimoorzaeem

I am not sure what to do with outstanding test coverage issues. This is about delete and deleteIO not being used. And indeed - right now there is no explicit cache entry removal anywhere. These functions are meant to be used in case of claims validation errors.

The question is: how to handle invalid JWTs? There are several options:

  1. Do not cache invalid JWTs at all
  2. Do not cache JWTs that cannot be parsed or have invalid signature. Cache all other JWTs, also JWTs containing invalid claims.
  3. Cache all token parsing and signature validation results - this is currently implemented option (ie. the type of values in the cache is Either Error (JSON.Object)
  4. Cache all tokens that can be parsed (but possibly having invalid signature).

I've decided option three makes sense as it also speeds up repeated invalid token requests. OTOH - it opens up possibility of cache filling attacks using randomly generated tokens. It is disputable if that is a bigger problem than overloading CPU with eg. JWTs signed with random key.

It looks to me option 4 would be the best compromise because it would prevent filling up cache with random garbage but would offload signature validation. I haven't implemented it as I don't know how to split parsing and signature validation with JOSE.

WDYT?

@wolfgangwalther

Copy link
Copy Markdown
Member

It looks to me option 4 would be the best compromise because it would prevent filling up cache with random garbage but would offload signature validation. I haven't implemented it as I don't know how to split parsing and signature validation with JOSE.

Parsing a JWT is really simple. Split on ., decode the first and second part as base64 - the result are two JSON objects, one the header, the other the payload.

The parser that is used for jose-jwt is here: https://hackage.haskell.org/package/jose-jwt-0.10.0/docs/src/Jose.Internal.Parser.html#jwt - it's internal, so I don't think you can use it.

But do we really need to? Parsing a JWT is really simple - but creating parseable tokens, with invalid signatures is just as easy. So I don't really see the additional value of option 4 vs option 3. If somebody wants to fill the cache, they can do so easily.

@mkleczek

mkleczek commented May 16, 2025

Copy link
Copy Markdown
Collaborator Author

Very interesting! 👀 👀

It also provides usage stats (ie. hit ratio, evictions count, size)

Maybe we could expose those as metrics.

@steve-chavez

Done. Three new counters are provided:

  • total number of cache lookups
  • total number of cache hits
  • total number of cache evictions

@mkleczek

mkleczek commented May 17, 2025

Copy link
Copy Markdown
Collaborator Author

But do we really need to? Parsing a JWT is really simple - but creating parseable tokens, with invalid signatures is just as easy. So I don't really see the additional value of option 4 vs option 3. If somebody wants to fill the cache, they can do so easily.

Indeed - any random bytes are a signature that consumes CPU to validate.

So that leaves us with the choice between caching negative results or caching only valid JWTs.

I've made cache implementation polymorphic over value computation monad so it is easy to change strategies. The latest commits introduce possibility of selecting one of two variants in PostgREST.Auth.JwtCache - it is a matter of changing cachingErrors to notCachingErrors in newJwtCache. For now I've left both variants and disabled unused local bindings warning to quiet linter. Once we decide which one is better we can remove the other one (or we can decide to make it configurable and leave both).

@wolfgangwalther

Copy link
Copy Markdown
Member

So that leaves us with the choice between caching negative results or caching only valid JWTs.

I wonder whether we can cache all valid JWTs, but including expired ones? Aka when validation fails because of expiry, still cache. When validation fails because of something else, don't.

The assumption is, that:

  • In regular operation the only real failure case that can happen regularly is expiry. We should stay fast for that.
  • If somebody wants to overload the system, they will find ways - or put differently: Just because we cache or don't cache invalid JWTs, attacks are not impossible. You'll need a different layer of protection against that anyway.

@mkleczek

Copy link
Copy Markdown
Collaborator Author

So that leaves us with the choice between caching negative results or caching only valid JWTs.

I wonder whether we can cache all valid JWTs, but including expired ones? Aka when validation fails because of expiry, still cache. When validation fails because of something else, don't.

That’s exactly option 2. Which is currently implemented as notCachingErrors variant (switchable in JwtCache.init).

As described in the description of the PR (first comment) - we don’t cache AuthResults but raw parsed claims that are re-validated for each request.

@wolfgangwalther

Copy link
Copy Markdown
Member

That’s exactly option 2.

Well, not exactly, but close enough I agree. We'd still cache those that fail, for example, and audience check or so. But that's totally fine, yes.

@mkleczek

mkleczek commented May 17, 2025

Copy link
Copy Markdown
Collaborator Author

That’s exactly option 2.

Well, not exactly, but close enough I agree. We'd still cache those that fail, for example, and audience check or so. But that's totally fine, yes.

Right - it is not exactly the same.

The reason I decided to leave claims checking until after cache lookup are two-fold:

  1. Time sensitive claims validation is dependent on well... time :) Not only exp but also for example nbf - it might become valid in the future.
  2. On the other hand aud validation is configuration sensitive and I wanted to minimize the number of configuration options which require whole cache reset when changed.

And, of course, claims checking is very fast, so there is little sense in caching it.

@wolfgangwalther

Copy link
Copy Markdown
Member

I wanted to minimize the number of configuration options which require whole cache reset when changed.

Yes, this makese a lot of sense!

So the only change that requires a reset is the change of secret, right?

@mkleczek

mkleczek commented May 17, 2025

Copy link
Copy Markdown
Collaborator Author

I wanted to minimize the number of configuration options which require whole cache reset when changed.

Yes, this makese a lot of sense!

So the only change that requires a reset is the change of secret, right?

That, and - of course - turning off caching alltogether (ie. setting jwt-cache-max-size=0)

@mkleczek

Copy link
Copy Markdown
Collaborator Author

@steve-chavez @wolfgangwalther @taimoorzaeem

After some more work on this PR I think it is now in a mergeable state (pending documentation changes, changelog adjustments etc. - and of course code review).

I am pretty confident it is working fine as I've added JWT cache behavior tests that verify hits/misses and evictions using metrics.

Please, let me know if there are any adjustments / changes required (or if you think the whole idea is wrong).

Comment thread src/PostgREST/Config.hs Outdated
Comment on lines +31 to +37
let auth = genToken [json|{"exp": 9999999999, "role": "postgrest_test_author", "id": "jdoe1"}|]

expectCounters
[
requests (+ 1)
, hits (+ 0)
] $

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really elegant way to test the metrics 💯 So easy to read!

Comment thread test/spec/Feature/Auth/JwtCacheSpec.hs Outdated
Comment thread test/spec/Feature/Auth/JwtCacheSpec.hs
Comment thread src/PostgREST/CLI.hs Outdated
Comment on lines +206 to +207
|## Enables JWT Cache and sets its max size, disables caching with 0
|# jwt-cache-max-size = 0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make the default 1000. That means the cache will be enabled by default for next major.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1000 is a rough estimation, mentioned before on #3802 (comment)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make the default 1000. That means the cache will be enabled by default for next major.

#4084 (comment)

Comment thread postgrest.cabal
@steve-chavez

steve-chavez commented May 28, 2025

Copy link
Copy Markdown
Member

Done. Three new counters are provided:
total number of cache lookups
total number of cache hits
total number of cache evictions

@mkleczek Awesome! How about a gauge for number of cached JWTs? I'm currently load testing the feature and that would help me ensure the cache size is maxed and that it goes down. This might be good for tests too?


I noticed that the previous JWT purge had a memory usage problem (#3889 (comment)) and this is now gone 🚀

I've recorded a video using postgrest-benchmark's OPTIONSUniqueJWT.js (this has the same logic as our jwt loadtest, but on dedicated hardware and using k6 instead of vegeta). Here you can see the memory reaches ~20% tops and then goes down to ~12%. Note: PostgREST is on a t3a.nano, so only has 0.5 GB.

Screencast.from.05-27-2025.09.56.23.PM.webm

Sharing the run results here for completeness:

     data_received..................: 51 MB  1.6 MB/s
     data_sent......................: 150 MB 4.8 MB/s
     http_req_blocked...............: avg=3.28µs  min=950ns    med=2.5µs    max=4.61ms  p(90)=3.17µs  p(95)=3.66µs 
     http_req_connecting............: avg=370ns   min=0s       med=0s       max=4.25ms  p(90)=0s      p(95)=0s     
   ✓ http_req_duration..............: avg=1.05ms  min=317.36µs med=961.44µs max=15.6ms  p(90)=1.55ms  p(95)=1.86ms 
       { expected_response:true }...: avg=1.05ms  min=317.36µs med=961.44µs max=15.6ms  p(90)=1.55ms  p(95)=1.86ms 
   ✓ http_req_failed................: 0.00%  ✓ 0           ✗ 254542
     http_req_receiving.............: avg=32.9µs  min=11.93µs  med=29.64µs  max=7.91ms  p(90)=45.02µs p(95)=49.7µs 
     http_req_sending...............: avg=16.81µs min=7.42µs   med=14.05µs  max=4.42ms  p(90)=28.89µs p(95)=32.53µs
     http_req_tls_handshaking.......: avg=0s      min=0s       med=0s       max=0s      p(90)=0s      p(95)=0s     
     http_req_waiting...............: avg=1ms     min=279.8µs  med=912.3µs  max=15.55ms p(90)=1.5ms   p(95)=1.8ms  
     http_reqs......................: 254542 8156.901769/s
     iteration_duration.............: avg=1.16ms  min=434.1µs  med=1.06ms   max=1.1s    p(90)=1.66ms  p(95)=1.97ms 
     iterations.....................: 254542 8156.901769/s
     vus............................: 10     min=0         max=10  
     vus_max........................: 10     min=10        max=10  

Also the metrics after the run (counters are high because I did some previous runs):

curl localhost:3001/metrics
# HELP pgrst_jwt_cache_evictions_total The total number of JWT cache evictions
# TYPE pgrst_jwt_cache_evictions_total counter
pgrst_jwt_cache_evictions_total 184905.0
# HELP pgrst_jwt_cache_hits_total The total number of JWT cache hits
# TYPE pgrst_jwt_cache_hits_total counter
pgrst_jwt_cache_hits_total 2817.0
# HELP pgrst_jwt_cache_requests_total The total number of JWT cache lookups
# TYPE pgrst_jwt_cache_requests_total counter
pgrst_jwt_cache_requests_total 188758.0

@mkleczek I've noticed that the jwt loadtest results show a perf drop compared to main and latest version. Is that expected?

@mkleczek

mkleczek commented May 28, 2025

Copy link
Copy Markdown
Collaborator Author

@mkleczek I've noticed that the jwt loadtest results show a perf drop compared to main and latest version. Is that expected?

Our load test is the worst possible case for this (and I would say: any bounded) cache: all JWTs are different so no caching but:

  • additional two hash map lookups (one for cache miss, another to put entry into the cache)
  • if the cache is smaller than the number of requests then we pay the price of eviction and garbage collection
  • updating metrics

In other words - cache thrashing at its best :)

What's more: in case of symmetric JWT keys I don't think cache lookup is faster than simply performing JWT verification.

Changes:
1. Refactoring and some cleanup of JWT handling code:
* Instead of caching AuthResult cache decoded claims (which signature was verified). Validating claims and determining role is done after cache lookup
* Cleaned up API so that usage of it is simplified: lookupJwtCache cache key >>= parseClaims configJwtAud time
* Handling of JwtCacheState initialization and updates of configuration is encapsulated in Auth.JwtCache module

2. Generic high performance (hopefully) scalable, dynamically resizeable cache implementation based on stm, stm-hamt and sieve algorithm. It also integrates with PostgREST measurements infrastructure providing usage stats (ie. hit ratio, evictions count)
@mkleczek mkleczek reopened this Jul 29, 2025
docs: reorganize sections to better explain caching

@steve-chavez steve-chavez left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work! 💯 . Let's merge now 🚀

@steve-chavez
steve-chavez merged commit 77ff11d into PostgREST:main Jul 29, 2025
35 checks passed
@steve-chavez

Copy link
Copy Markdown
Member

🙌 We're now included in the "Adoption" section of https://cachemon.github.io/SIEVE-website/ 🙌

taimoorzaeem added a commit to taimoorzaeem/postgrest that referenced this pull request Nov 8, 2025
This should reduce setup time for build process.

- cache: introduced in PostgREST#2928, defunct since PostgREST#4084
- clock: introduced in PostgREST#2928, defunct since PostgREST#4084
- heredoc: introduced in PostgREST#714, defunct since PostgREST#4390
- iproute: introduced in PostgREST#3560, defunct since PostgREST#4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
steve-chavez pushed a commit that referenced this pull request Nov 9, 2025
This should reduce setup time for build process.

- cache: introduced in #2928, defunct since #4084
- clock: introduced in #2928, defunct since #4084
- heredoc: introduced in #714, defunct since #4390
- iproute: introduced in #3560, defunct since #4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
postgrest-ci Bot pushed a commit that referenced this pull request Nov 10, 2025
This should reduce setup time for build process.

- cache: introduced in #2928, defunct since #4084
- clock: introduced in #2928, defunct since #4084
- heredoc: introduced in #714, defunct since #4390
- iproute: introduced in #3560, defunct since #4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 9921743)
wolfgangwalther pushed a commit that referenced this pull request Nov 10, 2025
This should reduce setup time for build process.

- cache: introduced in #2928, defunct since #4084
- clock: introduced in #2928, defunct since #4084
- heredoc: introduced in #714, defunct since #4390
- iproute: introduced in #3560, defunct since #4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 9921743)
taimoorzaeem added a commit to taimoorzaeem/postgrest that referenced this pull request Nov 19, 2025
Removes a test related to jwt cache which is stale
since PostgREST#4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
steve-chavez pushed a commit that referenced this pull request Nov 19, 2025
Removes a test related to jwt cache which is stale
since #4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
postgrest-ci Bot pushed a commit that referenced this pull request Nov 19, 2025
Removes a test related to jwt cache which is stale
since #4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 379eaec)
steve-chavez pushed a commit that referenced this pull request Nov 19, 2025
Removes a test related to jwt cache which is stale
since #4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
(cherry picked from commit 379eaec)
taimoorzaeem added a commit to taimoorzaeem/postgrest that referenced this pull request Mar 9, 2026
`PGRST_JWT_CACHE_MAX_LIFETIME` is defunct since PostgREST#4084 is merged.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
taimoorzaeem added a commit to taimoorzaeem/postgrest that referenced this pull request Mar 9, 2026
`PGRST_JWT_CACHE_MAX_LIFETIME` is defunct since PostgREST#4084 is merged.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
steve-chavez pushed a commit that referenced this pull request Mar 9, 2026
`PGRST_JWT_CACHE_MAX_LIFETIME` is defunct since #4084 is merged.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
laurenceisla pushed a commit to supabase/postgrest that referenced this pull request May 28, 2026
This should reduce setup time for build process.

- cache: introduced in PostgREST#2928, defunct since PostgREST#4084
- clock: introduced in PostgREST#2928, defunct since PostgREST#4084
- heredoc: introduced in PostgREST#714, defunct since PostgREST#4390
- iproute: introduced in PostgREST#3560, defunct since PostgREST#4288

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
laurenceisla pushed a commit to supabase/postgrest that referenced this pull request May 28, 2026
Removes a test related to jwt cache which is stale
since PostgREST#4084.

Signed-off-by: Taimoor Zaeem <taimoorzaeem@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants