Skip to content

parser_ltsv: optimize key-value parsing by removing redundant string scan#5353

Merged
kenhys merged 1 commit into
fluent:masterfrom
Watson1978:parser_ltsv
May 7, 2026
Merged

parser_ltsv: optimize key-value parsing by removing redundant string scan#5353
kenhys merged 1 commit into
fluent:masterfrom
Watson1978:parser_ltsv

Conversation

@Watson1978

@Watson1978 Watson1978 commented May 6, 2026

Copy link
Copy Markdown
Contributor

Which issue(s) this PR fixes:
Fixes #

What this PR does / why we need it:
Remove the redundant String#include? check before String#split in the LTSV parser.
Previously, pair.include?(@label_delimiter) scanned the string once,
and then pair.split(@label_delimiter, 2) scanned it again.

Microbenchmark

require 'bundler/inline'
gemfile do
  source 'https://rubygems.org'
  gem 'benchmark-ips'
end

message = "time:2013/02/28 12:00:00\thost:192.168.0.1\treq_id:111"

Benchmark.ips do |x|
  x.report('before') do
    r = {}
    message.split("\t").each do |pair|
      if pair.include? ":"
        key, value = pair.split(":", 2)
        r[key] = value
      end
    end
  end

  x.report('after') do
    r = {}
    message.split("\t").each do |pair|
      key, value = pair.split(":", 2)
      r[key] = value if value
    end
  end

  x.compare!
end
ruby 4.0.3 (2026-04-21 revision 85ddef263a) +PRISM [x86_64-linux]
Warming up --------------------------------------
              before   114.156k i/100ms
               after   128.758k i/100ms
Calculating -------------------------------------
              before      1.140M (± 0.1%) i/s  (877.32 ns/i) -      5.708M in   5.007564s
               after      1.285M (± 0.1%) i/s  (777.98 ns/i) -      6.438M in   5.008577s

Comparison:
               after:  1285378.0 i/s
              before:  1139837.2 i/s - 1.13x  slower

Docs Changes:
N/A

Release Note:

  • parser_ltsv: optimize key-value parsing by removing redundant string scan

…scan

Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
@Watson1978 Watson1978 added this to the v1.20.0 milestone May 6, 2026
@Watson1978 Watson1978 requested a review from kenhys May 6, 2026 06:14

@kenhys kenhys left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

@kenhys kenhys merged commit 4b7b6b2 into fluent:master May 7, 2026
35 of 37 checks passed
@Watson1978 Watson1978 deleted the parser_ltsv branch May 7, 2026 03:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants