Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ingest/ledgerbackend: Add ledger backend which replays ledgers #5584

Merged
merged 9 commits into from
Feb 6, 2025

Conversation

tamirms
Copy link
Contributor

@tamirms tamirms commented Jan 30, 2025

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've reviewed the changes in this PR and if I consider them worthwhile for being mentioned on release notes then I have updated the relevant CHANGELOG.md within the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

Close #5480

Following #5556 , this PR implements a ledger backend which takes synthetically generated ledgers, merges them with real ledgers, and then replays them at a configurable rate.

Why

This ledger backend will be used to benchmark ingestion duration for horizon / rpc when ingesting ledgers with a high volume of soroban transactions. These benchmarks will inform us of horizon and rpc's ceiling when it comes to ingestion throughput.

Known limitations

[N/A]

@tamirms tamirms marked this pull request as ready for review January 31, 2025 19:59
@tamirms tamirms requested a review from a team January 31, 2025 20:04
@tamirms
Copy link
Contributor Author

tamirms commented Feb 3, 2025

@karthikiyer56 I think I have addressed your code review feedback, could you take another look?

ingest/replay_backend.go Outdated Show resolved Hide resolved
ingest/replay_backend.go Outdated Show resolved Hide resolved
ingest/replay_backend.go Outdated Show resolved Hide resolved
ingest/replay_backend.go Outdated Show resolved Hide resolved
ingest/replay_backend.go Outdated Show resolved Hide resolved
ingest/replay_backend.go Outdated Show resolved Hide resolved
@2opremio
Copy link
Contributor

2opremio commented Feb 3, 2025

All my comments are optional. I am mostly concerned about the role of the passed ledger backend, which is a bit unintuitive to me.

@tamirms tamirms merged commit 3f84951 into stellar:master Feb 6, 2025
23 checks passed
@tamirms tamirms deleted the replay-backend branch February 6, 2025 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement a LedgerBackend for benchmarking ingestion performance as a function of ledger size
3 participants