services/horizon: Update asset stats processor to ingest new restoration meta #5611
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Checklist
PR Structure
otherwise).
services/friendbot
, orall
ordoc
if the changes are broad or impact manypackages.
Thoroughness
.md
files, etc... affected by this change). Take a look in the
docs
folder for a given service,like this one.
Release planning
CHANGELOG.md
within the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.
What
Fixes #5422 , #5599
This PR updates the asset stats processor to handle the new restoration meta which will be introduced in the upcoming stellar-core v23 release (see stellar/stellar-core#4623 ). The protocol 23 release will also implement eviction of archived ledger entries. The only ingestion processor in Horizon which ingests ledger entries which can be archived / evicted is the asset stats processor.
Now, we no longer store asset stats for archived contract balances. Instead, we immediately delete contract balances from the horizon db as soon as the ttl expires. This allows us to ignore the eviction meta as the ledger entry no longer exists in the horizon db by the time it is evicted.
This PR also adds a runtime check to assert that the stellar-core binary supports protocol 23. This check is necessary because we don't want to maintain support for both protocol 23 and protocol 21 / 22 versions of restoration meta (see #5599 for more details).
Known limitations
There is no integration test which exercises eviction because we still need a stellar rpc docker image which supports protocol 23 before we can run protocol 23 integration tests. Once that docker image is available, we create another PR that adds an eviction integration test.
However, the restoration meta is still exercised in the integration tests because I have updated the integration tests to use the experimental core build ( stellar/unsafe-stellar-core:22.1.1-2291.a50f3f919.focal-do-not-use-in-prd ) which implements the new restoration meta in all protocol versions.