Skip to content

Conversation

rescrv
Copy link
Contributor

@rescrv rescrv commented Oct 16, 2025

Description of changes

We need to not fetch more than a minute or two into the future to make s3heap-service performant and
cost-effective. This PR makes it so that the s3heap will allow limiting how far into the future a
call will look for peek/prune.

Test plan

CI

Migration plan

N/A

Observability plan

N/A

Documentation Changes

N/A

Copy link

Reviewer Checklist

Please leverage this checklist to ensure your code review is thorough before approving

Testing, Bugs, Errors, Logs, Documentation

  • Can you think of any use case in which the code does not behave as intended? Have they been tested?
  • Can you think of any inputs or external events that could break the code? Is user input validated and safe? Have they been tested?
  • If appropriate, are there adequate property based tests?
  • If appropriate, are there adequate unit tests?
  • Should any logging, debugging, tracing information be added or removed?
  • Are error messages user-friendly?
  • Have all documentation changes needed been made?
  • Have all non-obvious changes been commented?

System Compatibility

  • Are there any potential impacts on other parts of the system or backward compatibility?
  • Does this change intersect with any items on our roadmap, and if so, is there a plan for fitting them together?

Quality

  • Is this code of a unexpectedly high quality (Readability, Modularity, Intuitiveness)

Copy link
Contributor

Add Time Cut-Off Limiting to s3heap Scans for Peek/Prune Operations

This pull request introduces an enhancement to the s3heap service by allowing callers to limit how far into the future the system will scan during peek and prune operations. The main goal is to improve performance and cost-effectiveness by avoiding unnecessary reads of future-scheduled tasks, which can mostly be ignored for current processing. The change adds a new optional time_cut_off field to the Limits struct and wires support for this through user-facing APIs, core scan logic, and documentation. Related unit tests are updated to ensure that the new limit is included in equality and construction checks.

Key Changes

• Added time_cut_off: Option<DateTime<Utc>> field to Limits struct in rust/s3heap/src/lib.rs to allow scan cut-off.
• Implemented with_time_cut_off() builder method on Limits and documented usage.
• Modified prune() and peek() methods to respect time_cut_off and exit scans early for buckets after the specified cut-off time.
• Updated all related API documentation, method comments, and examples to include the new cut-off limit.
• Extended and updated unit tests in rust/s3heap/tests/test_unit_tests.rs to account for the new time_cut_off field in equality and construction.

Affected Areas

rust/s3heap/src/lib.rs (limits definition, scanning logic, documentation)
rust/s3heap/tests/test_unit_tests.rs (unit test field inclusion)

This summary was automatically generated by @propel-code-bot

@rescrv rescrv requested a review from tanujnay112 October 16, 2025 20:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant