Skip to content

output: memoize calculate_timekey to reduce redundant calculations#5288

Merged
kenhys merged 1 commit into
fluent:masterfrom
Watson1978:output
Mar 23, 2026
Merged

output: memoize calculate_timekey to reduce redundant calculations#5288
kenhys merged 1 commit into
fluent:masterfrom
Watson1978:output

Conversation

@Watson1978

@Watson1978 Watson1978 commented Mar 21, 2026

Copy link
Copy Markdown
Contributor

Which issue(s) this PR fixes:
Fixes #

What this PR does / why we need it:
calculate_timekey is called for every emitted record.
Since logs usually arrive chronologically, consecutive records almost always fall into the same timekey.
However, the current implementation performs % and to_i for every single call.

When running rake benchmark:run:in_tail with 10 GB of data, this memoization improved the overall throughput:

Benchmark before after
rake benchmark:run:in_tail 43.170137 41.584905

Docs Changes:
N/A

Release Note:

  • output: Improve performance by memoizing calculate_timekey

Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
@Watson1978 Watson1978 requested a review from kenhys March 21, 2026 07:53
@Watson1978 Watson1978 added this to the v1.20.0 milestone Mar 21, 2026
@kenhys kenhys merged commit f261802 into fluent:master Mar 23, 2026
51 of 53 checks passed
@kenhys

kenhys commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

Thanks

@Watson1978 Watson1978 deleted the output branch March 24, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants