feat: grant spent amounts #3757

BlairCurrey · 2025-11-21T21:22:02Z

Parent branch for grant spent amounts calculation rework. Most of the code was already reviewed, but I did add something here that was not captured in the other issues. And that is getting the grant spent amounts in this edge case:

amount has been spent against grant
new rafiki version is deployed with this method for calculating spent amounts
GET /outgoing-payment-grant` is called (no spent amounts, but amount was spent against the grant)

In this case we are summing all historical payments (added that in this PR). We do the same thing on the outgoing payment create side so I shared the logic.

fixes #3369
linear https://linear.app/interledger/issue/RAF-1031/implement-new-grant-spent-amount-calculation

* feat: add grant spent amount table * fix: change interval start/end string to timestamp * feat: fix filename, update op grant table * fix: rm erroneously added migration to tsconfig

* feat: calculate grant spent amounts from new table * chore: linting * fix: revise grant spent amounts calc to track interval * fix: throw better error, dont inti interval amounts unless they exist * fix: linter, wip commented out code * refactor(backend): rename interval statuses * fix: ensure correct amount (interval/total) is used for grant spent amount * test(backend): grant spent amounts * fix(backend): make logic more typesafe, avoid having to defend w/ internal server error * test(backend): legacy grant calculations * test(backend): classify payment interval fn * test(backend): rm commented-out classify payment interval case * refactor(backend): improve semantics of interval helper fn * test(backend): add non-interval cases to create op * Apply suggestions from code review Co-authored-by: Max Kurapov <[email protected]> * refactor: simplifiy interval calc * chore: rm unused imports * refactor: validate grant payment logic * refactor: improve grant spent amount calc type safety * refactor: improve grant spent amount calc type safety * fix: missing trx * chore: handle bad interval state * chore: rm some superfluous comments * chore: format --------- Co-authored-by: Max Kurapov <[email protected]>

) * feat: calculate grant spent amounts from new table * chore: linting * fix: revise grant spent amounts calc to track interval * fix: throw better error, dont inti interval amounts unless they exist * fix: linter, wip commented out code * refactor(backend): rename interval statuses * fix: ensure correct amount (interval/total) is used for grant spent amount * test(backend): grant spent amounts * fix(backend): make logic more typesafe, avoid having to defend w/ internal server error * test(backend): legacy grant calculations * test(backend): classify payment interval fn * test(backend): rm commented-out classify payment interval case * refactor(backend): improve semantics of interval helper fn * test(backend): add non-interval cases to create op * Apply suggestions from code review Co-authored-by: Max Kurapov <[email protected]> * feat: partially handled grant spent amount scenarios on settle * feat: handle non-interval partial settleamount cases * test(backend): failure grant spent amount case for sucessive payment * test(backend): some grant calc interval cases * test(backend): counting grant payments across interval boundaries * chore(backend): rm commented out test * test(backend): new grant spent amount on payment completion * test(backend): improve interval grant calc test to show summation * test(backend): failure edge case * chore(backend): rm comment * chore(backend): fix lint errors * chore(backend): rm debug logs * test(backend): fix failing * Update packages/backend/src/open_payments/payment/outgoing/lifecycle.ts Co-authored-by: Max Kurapov <[email protected]> * fix(backend): rm extra spent amount record check * feat(backend): return debit amount from .pay * test(backend): failing test for grant spent amount bug - if 2 payments are create then 2 payments are processed, it assocaites the wrong spent amount with the wrong payment * fix(backend): grant spent amounts calc race conditions - partially implemented fix (missing interval stuff), not fully validated by tests yet * test(backend): grant calc race condition * fix(backend): grant spent amount race conditions with failure * chore(backend): format * test(backend): add failing interval race condition test * fix(backend): interval boundary/race condition edge case - edge case is when there are create/complete race conditions around interval boundaries * test(backend): improve edge case test - ensures we are testing that the latest interval amount is used as the base for interval amounts, not just the payment in the interval being completed * fix(backend): dont query for latest interval payment unecunnecessarily - not necessary if we are within the interval of the payment being completed * fix(backend): type mismatch * test(backend): payment retries do not add additional spent record * fix(backend): use correct debit amount for spent amount recalc * fix(backend): set correct payment state on grant spent amount updates * refactor(backend): dedupe revert/handle grant spent amounts - turned duplicated logic into shared functions - example fn where revert/update is all in one, but felt it was more complicated * chore: rm unused fn * test(backend): fix ilp pay return expectations * refactor(backend): move revert/update grant spent amounts to op service * chore: rm unused import * fix: handle grant spent amounts on payment cancellation * chore: cleanup commented out code * chore: rm submodule added in error * chore(backend): add error logs * fix(backend): explicit grant spent amount formation, more test assertions * refactor(backend): only return receive amt from .pay * fix(backend): build error --------- Co-authored-by: Max Kurapov <[email protected]>

netlify · 2025-11-21T21:22:09Z

✅ Deploy Preview for brilliant-pasca-3e80ec ready!

Name	Link
🔨 Latest commit	`99d5ed9`
🔍 Latest deploy log	https://app.netlify.com/projects/brilliant-pasca-3e80ec/deploys/6965573110969f0008825990
😎 Deploy Preview	https://deploy-preview-3757--brilliant-pasca-3e80ec.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

github-actions · 2025-11-21T21:27:14Z

🚀 Performance Test Results

Test Configuration:

VUs: 4
Duration: 1m0s

Test Metrics:

Requests/s: 43.72
Iterations/s: 14.59
Failed Requests: 0.00% (0 of 2631)

📜 Logs


> [email protected] run-tests:testenv /home/runner/work/rafiki/rafiki/test/performance
> ./scripts/run-tests.sh -e test "-k" "-q" "--vus" "4" "--duration" "1m"

Cloud Nine GraphQL API is up: http://localhost:3101/graphql
Cloud Nine Wallet Address is up: http://localhost:3100/
Happy Life Bank Address is up: http://localhost:4100/
cloud-nine-wallet-test-backend already set
cloud-nine-wallet-test-auth already set
happy-life-bank-test-backend already set
happy-life-bank-test-auth already set
     data_received..................: 950 kB 16 kB/s
     data_sent......................: 2.0 MB 34 kB/s
     http_req_blocked...............: avg=6.73µs   min=2.07µs   med=5.07µs   max=972.09µs p(90)=6.18µs   p(95)=6.76µs  
     http_req_connecting............: avg=239ns    min=0s       med=0s       max=140.81µs p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=90.8ms   min=7.76ms   med=74.07ms  max=580.67ms p(90)=155.68ms p(95)=176.78ms
       { expected_response:true }...: avg=90.8ms   min=7.76ms   med=74.07ms  max=580.67ms p(90)=155.68ms p(95)=176.78ms
     http_req_failed................: 0.00%  ✓ 0         ✗ 2631
     http_req_receiving.............: avg=94.6µs   min=26.96µs  med=79.94µs  max=2.93ms   p(90)=122.88µs p(95)=165.16µs
     http_req_sending...............: avg=35.88µs  min=10.13µs  med=26.62µs  max=3.22ms   p(90)=39.36µs  p(95)=54µs    
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=90.67ms  min=7.59ms   med=73.98ms  max=580.54ms p(90)=155.55ms p(95)=176.7ms 
     http_reqs......................: 2631   43.720445/s
     iteration_duration.............: avg=273.78ms min=170.49ms med=257.54ms max=1.11s    p(90)=337.14ms p(95)=374.12ms
     iterations.....................: 878    14.590099/s
     vus............................: 4      min=4       max=4 
     vus_max........................: 4      min=4       max=4

* feat: add /GET outgoing-payment-grant route * fix: change null state return to return null * fix: rm old comments * fix: add get grant spent amount service tests * fix: add get grant spent amount controller tests * fix(backend): linter * fix(backend): tenanted path * fix: dont require identifier in outgoing payment access item introspection * feat(backend): rm unecessary check * test(auth): add test * test(backend): add middleware test * refactor(backend): cleanup grant spent amount retrieval * chore: fix lint * test(backend): fix grant id check * test(backend): simplify logic, return 0 amount when able * refactor(backend): token introspection middleware - make 2nd introspection middleware for new route instead of generalizing exisitng middleware * test(backend): rm no longer needed test * fix: rm unused import * Update packages/backend/src/open_payments/auth/middleware.ts Co-authored-by: Max Kurapov <[email protected]> * fix: return 403 for non create grant for spent amounts used to return null spent amounts * fix: add logs, use luxon interval method * fix: scope of else block * Update packages/backend/src/open_payments/auth/middleware.ts Co-authored-by: Max Kurapov <[email protected]> * chore: fix name * feat: test for new middleware/shared fn * chore: make action required --------- Co-authored-by: Max Kurapov <[email protected]>

mkurapov

After we have updated the NodeJS client, we should also update an integration test to verify grant spent amounts as well

mkurapov · 2026-01-05T15:39:40Z

packages/backend/src/open_payments/auth/middleware.test.ts

+
+      await middleware(ctx, next)
+
+      expect(mockTokenIntrospectionClient.introspect).toHaveBeenCalledWith({


let's also validate that next was called heere

njlie

LGTM, seemed to work when I tested the local env a bit

BlairCurrey · 2026-01-12T20:17:34Z

packages/backend/migrations/20250409173617_add_outgoing_payment_grant_spent_amounts_table.js

+    })
+    .then(() => {
+      return knex.raw(
+        'CREATE INDEX outgoingPaymentGrantSpentAmounts_grantId_createdAt_desc_idx ON "outgoingPaymentGrantSpentAmounts" ("grantId", "createdAt" DESC)'


I think I'll leave it for now.... but I was thinking about changing this:

'CREATE INDEX outgoingPaymentGrantSpentAmounts_grantId_createdAt_desc_idx ON "outgoingPaymentGrantSpentAmounts" ("grantId", "createdAt" DESC)'

To include the interval fields:

knex.raw('CREATE INDEX ... ON "outgoingPaymentGrantSpentAmounts" ("grantId", "intervalStart", "intervalEnd", "createdAt" DESC)')`

because we look it up like this in the worker when processing the payment:

await OutgoingPaymentGrantSpentAmounts.query(deps.knex) .where('grantId', grantId) .where('intervalStart', latestPaymentSpentAmounts.intervalStart) .where('intervalEnd', latestPaymentSpentAmounts.intervalEnd) .orderBy('createdAt', 'desc') .first()

Dont have great intuition on tradeoffs. I guess it would slow writes some amount in all cases and speed up reads some amount in a subset of cases (when we lookup by interval - only in worker I think). Probably no db storage space difference since a created_at index is going to already lead to a unique index entry per row. Overall I guess it feels like a bad idea. I guess we probably read a bit more (read and write on create, read and write in worker, read in GET /outgoing-payment-grant) but we only lookup by interval in one place.

BlairCurrey · 2026-01-12T20:22:02Z

Made a few small tweaks:

recorded metric for those cases where we log an error due to unexpected state but dont through
fixed a few spots where we cast from BigInt -> Number -> BigInt to use Math.max but lost precision in doing so

BlairCurrey and others added 7 commits April 10, 2025 14:29

feat: add grant spent amount table (#3389)

4374044

* feat: add grant spent amount table * fix: change interval start/end string to timestamp * feat: fix filename, update op grant table * fix: rm erroneously added migration to tsconfig

Merge branch 'main' into bc/raf-1031/grant-spent-amounts

f9ad11f

Merge branch 'main' into bc/raf-1031/grant-spent-amounts

06e3acf

Merge branch 'main' into bc/raf-1031/grant-spent-amounts

eacf1c5

fix: test to use tenantid

bb46de4

github-actions bot added type: tests Testing related pkg: backend Changes in the backend package. type: source Changes business logic labels Nov 21, 2025

BlairCurrey changed the title ~~Bc/raf 1031/grant spent amounts~~ feat: grant spent amounts Nov 21, 2025

BlairCurrey and others added 4 commits November 24, 2025 13:26

fix: update lifecycle tests for mt

eed6d10

feat: use new open payments spec version

23756ed

fix: failing lifecycle tests

e637544

github-actions bot added the pkg: auth Changes in the GNAP auth package. label Dec 11, 2025

BlairCurrey added 7 commits December 11, 2025 12:25

Merge branch 'main' into bc/raf-1031/grant-spent-amounts

6608da7

fix: import

1371fb9

chore: fix typo in bruno folder

c94cf27

fix: calculate legacy path in get spent amounts

a7c3b38

refactor: simplifiy get grant spent amounts logic

ddbe537

fix: migrate legacy payments

465e92b

fix: use trx

bf27169

BlairCurrey marked this pull request as ready for review December 16, 2025 03:03

BlairCurrey requested review from cozminu, mkurapov, njlie and oana-lolea December 16, 2025 03:03

BlairCurrey requested review from koekiebox and sanducb December 16, 2025 03:03

mkurapov previously approved these changes Jan 5, 2026

View reviewed changes

test(backend): ensure middleware next is called

0214779

BlairCurrey dismissed mkurapov’s stale review via 0214779 January 6, 2026 20:48

Merge branch 'main' into bc/raf-1031/grant-spent-amounts

1b8d896

njlie previously approved these changes Jan 9, 2026

View reviewed changes

BlairCurrey added 2 commits January 12, 2026 14:43

fix(backend): fix potential precision loss

457478b

feat: add metric to unexpected state (that doesnt error)

99d5ed9

BlairCurrey commented Jan 12, 2026

View reviewed changes

BlairCurrey dismissed njlie’s stale review via 99d5ed9 January 12, 2026 20:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: grant spent amounts #3757

feat: grant spent amounts #3757

Uh oh!

BlairCurrey commented Nov 21, 2025 •

edited

Loading

Uh oh!

netlify bot commented Nov 21, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 21, 2025 •

edited

Loading

Uh oh!

mkurapov left a comment

Uh oh!

mkurapov Jan 5, 2026

Uh oh!

njlie left a comment

Uh oh!

BlairCurrey Jan 12, 2026

Uh oh!

BlairCurrey commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants


		await middleware(ctx, next)

		expect(mockTokenIntrospectionClient.introspect).toHaveBeenCalledWith({

feat: grant spent amounts #3757

Are you sure you want to change the base?

feat: grant spent amounts #3757

Uh oh!

Conversation

BlairCurrey commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for brilliant-pasca-3e80ec ready!

Uh oh!

github-actions bot commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Performance Test Results

Uh oh!

mkurapov left a comment

Choose a reason for hiding this comment

Uh oh!

mkurapov Jan 5, 2026

Choose a reason for hiding this comment

Uh oh!

njlie left a comment

Choose a reason for hiding this comment

Uh oh!

BlairCurrey Jan 12, 2026

Choose a reason for hiding this comment

Uh oh!

BlairCurrey commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

BlairCurrey commented Nov 21, 2025 •

edited

Loading

netlify bot commented Nov 21, 2025 •

edited

Loading

github-actions bot commented Nov 21, 2025 •

edited

Loading