EC container quota is not freed after objects are expired

[evgeniiz321]

PR - nspcc-dev/neofs-testcases#1255

Report - https://rest.fs.neo.org/Aku38wpn2xEqfEPxmwyAa7N3VnAkMV8PEVkpLqwKM1z3/1591-1766811866/index.html#suites/cc4ee90de49dc664c89450ba41df7ecb/b2f739cd24fcdb2/

Test - test_ec_container_object_lifetime_and_quota_reclaim

Scenario -

1. Put a couple objects with lifetime=1 to a EC container:

COMMAND: ./neofs-cli --config /home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/wallet_config.yml object put --rpc-endpoint 'localhost:50175' --wallet '/home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/test-run-2025-12-27-03-35-36-359236/wallet-b02e5b50-73ba-4cbe-813b-63aabe708f03.json' --cid 'H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR' --file '/home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/test-run-2025-12-27-03-35-36-359236/TestFilesDir/temp_file_daf0ba54-fad7-4bfb-a4c1-942b27bae298' --lifetime 1 --no-progress --timeout '180s'
RETCODE: 0

STDOUT:
[/home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/test-run-2025-12-27-03-35-36-359236/TestFilesDir/temp_file_daf0ba54-fad7-4bfb-a4c1-942b27bae298] Object successfully stored
  OID: 5d8Cs4Kb1rm5HJrG3uj1EU1REiLH7omfuy3tpMjcG5Nz
  CID: H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR

STDERR:

Start / End / Elapsed	 04:29:33.562908 / 04:29:34.300639 / 0:00:00.737731

2. Wait a couple epochs and some additional time, check quota reports:

COMMAND: ./neofs-adm --config /home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/test-run-2025-12-27-03-35-36-359236/env_files/neofs-env-2025-12-27-03-35-38-139986427792256/inner_ring_lfuyonejcb/ir_network_config_ydcssdemsl.yml fschain load-report --rpc-endpoint 'http://localhost:55313' --cid 'H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR'
RETCODE: 0

STDOUT:
Report #0:
  Reporter's pubic Key: b3257e3a110ccb387e4ae0f4b646bc7d7df8c1ab686a06f08cef1b2a434c66d982ed25074cc0247ef19769ccba64ad69aa96f03337db467f6d79abaa790d7d77:
  Size: 44
  Objects: 0
  Update epoch: 141
  Reports number: 1

Report #1:
  Reporter's pubic Key: fda93cf61b3be6c1e8e366236675e69e51a963b5f1b47558aa0a502f13370236d06f2941e6818fad986f00479d3ad620690d4e9e7b3db1fa3913010c874cbe01:
  Size: 44
  Objects: 0
  Update epoch: 141
  Reports number: 1

Report #2:
  Reporter's pubic Key: 99694a57668cbfbfbcf04e60976f0e883b245ae1eb07b7a5fb92c84f7bbc45eeee301ec32fe7eb94507edd7f34555ae6a428ec5ce421a14a47130dd015997486:
  Size: 44
  Objects: 0
  Update epoch: 141
  Reports number: 1

Report #3:
  Reporter's pubic Key: 46c5e4a43fe6a3a63c35a9876f14aa2cbfcf6e1ebd2fb43b844631bdf30e296f77bae4da94f7ffae8d38f3cbcdccee37f39c66ba96bf52e83cec0b63918aba23:
  Size: 44
  Objects: 0
  Update epoch: 141
  Reports number: 1

STDERR:

Start / End / Elapsed	 04:30:02.201617 / 04:30:02.221033 / 0:00:00.019416

No objects reported.
3. Try to put a new object into the ec container (object size is the same as in 1, more precisely - quota is 200bytes, object size is 80):

COMMAND: ./neofs-cli --config /home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/wallet_config.yml object put --rpc-endpoint 'localhost:50175' --wallet '/home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/test-run-2025-12-27-03-35-36-359236/wallet-b02e5b50-73ba-4cbe-813b-63aabe708f03.json' --cid 'H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR' --file '/home/runner/work/neofs-testcases/neofs-testcases/neofs-testcases/test-run-2025-12-27-03-35-36-359236/TestFilesDir/temp_file_95844a9d-d7d2-43e2-b89e-3dba0cde7099' --no-progress --timeout '180s'
RETCODE: 1

STDOUT:
Error: rpc error: finish object stream: status: code = 2054 message = size quota limits are exceeded

STDERR:

Start / End / Elapsed	 04:30:02.222857 / 04:30:02.679014 / 0:00:00.456157

Logs:

neofs env files-60.zip

[roman-khimov]

#3717?

[cthulhu-rider]

2025-12-27T04:29:45.560Z	info	log/log.go:12	local object storage operation	{"shard_id": "YAgbxbbVKF3XDJK1LnTbnj", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/5d8Cs4Kb1rm5HJrG3uj1EU1REiLH7omfuy3tpMjcG5Nz", "op": "metabase DELETE"}
2025-12-27T04:29:45.561Z	info	log/log.go:12	local object storage operation	{"shard_id": "WNbwWiGDTjgYnB8vAuieSx", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/5d8Cs4Kb1rm5HJrG3uj1EU1REiLH7omfuy3tpMjcG5Nz", "op": "metabase DELETE"}
2025-12-27T04:29:45.561Z	info	log/log.go:12	local object storage operation	{"shard_id": "YAgbxbbVKF3XDJK1LnTbnj", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/78mduoJbsqY2MXdPkERF1m3YFmVhmXcf4PBL75zPKtqX", "op": "metabase DELETE"}
2025-12-27T04:29:45.561Z	info	log/log.go:12	local object storage operation	{"shard_id": "WNbwWiGDTjgYnB8vAuieSx", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/78mduoJbsqY2MXdPkERF1m3YFmVhmXcf4PBL75zPKtqX", "op": "metabase DELETE"}
...
2025-12-27T05:04:01.513Z	info	log/log.go:12	local object storage operation	{"shard_id": "YAgbxbbVKF3XDJK1LnTbnj", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/5d8Cs4Kb1rm5HJrG3uj1EU1REiLH7omfuy3tpMjcG5Nz", "op": "metabase DELETE"}
2025-12-27T05:04:01.513Z	info	log/log.go:12	local object storage operation	{"shard_id": "WNbwWiGDTjgYnB8vAuieSx", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/5d8Cs4Kb1rm5HJrG3uj1EU1REiLH7omfuy3tpMjcG5Nz", "op": "metabase DELETE"}
2025-12-27T05:04:01.514Z	info	log/log.go:12	local object storage operation	{"shard_id": "YAgbxbbVKF3XDJK1LnTbnj", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/78mduoJbsqY2MXdPkERF1m3YFmVhmXcf4PBL75zPKtqX", "op": "metabase DELETE"}
2025-12-27T05:04:01.514Z	info	log/log.go:12	local object storage operation	{"shard_id": "WNbwWiGDTjgYnB8vAuieSx", "address": "H2t1NFAfezqmA46qDjUJTeQeLXQAgXVcHq4q3ZM2EefR/78mduoJbsqY2MXdPkERF1m3YFmVhmXcf4PBL75zPKtqX", "op": "metabase DELETE"}

regular but futile attempts to remove objects. They are stuck

we already observed similar symptoms in #3715. It is not yet clear whether there is a connection b/w problems

cannot reproduce this in devenv w/ the latest version. @evgeniiz321 which one was used in test?

[cthulhu-rider]

#3717?

guess it may lead to

Report #0:
  Reporter's pubic Key: b3257e3a110ccb387e4ae0f4b646bc7d7df8c1ab686a06f08cef1b2a434c66d982ed25074cc0247ef19769ccba64ad69aa96f03337db467f6d79abaa790d7d77:
  Size: 44
  Objects: 0

when EC parent is cleaned, payload counter is unchanged while object one is decremented

[cthulhu-rider]

@evgeniiz321 pls try with the latest SN and reset

neofs-node/config/example/node.yaml

Line 72 in 24a99c2

quota_ttl: 1m # quotas limit cache TTL value. Negative values disables caching. Default value: 10m.

quota_ttl: -1