Skip to content

Commit 0547ff9

Browse files
authored
parser_csv: skip empty or unparseable lines (#5355)
**Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: This PR fixes an issue in `parser_csv` where processing an empty line causes a `TypeError`. When `parser_csv` receives an empty line, `CSV.parse_line` returns `nil`. Currently, passing this `nil` value to `Array#zip` throws a `TypeError: wrong argument type NilClass (must respond to :each)`. ### Reproduce config: ``` <source> @type tail path "#{File.expand_path '~/tmp/fluentd/test.csv'}" tag test.csv read_from_head true <parse> @type csv keys k1,k2,k3 </parse> </source> <match test.csv> @type stdout </match> ``` Prepare input data: ``` $ echo "A,B,C" > ~/tmp/fluentd/test.csv $ echo "" >> ~/tmp/fluentd/test.csv $ echo "D,E,F" >> ~/tmp/fluentd/test.csv ``` Result of before changing: ``` 2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv 2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found 2026-05-09 17:23:14 +0900 [warn]: #0 this parameter is highly recommended to save the position to resume tailing. 2026-05-09 17:23:14 +0900 [info]: #0 starting fluentd worker pid=41671 ppid=41646 worker=0 2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv 2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found file="/home/watson/tmp/fluentd/test.csv" line="" error="wrong argument type NilClass (must respond to :each)" 2026-05-09 17:23:14.692710744 +0900 test.csv: {"k1":"A","k2":"B","k3":"C"} 2026-05-09 17:23:14.692801343 +0900 test.csv: {"k1":"D","k2":"E","k3":"F"} 2026-05-09 17:23:14 +0900 [info]: #0 fluentd worker is now running worker=0 ``` **Docs Changes**: N/A **Release Note**: * parser_csv: skip empty or unparseable lines Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
1 parent 4b7b6b2 commit 0547ff9

2 files changed

Lines changed: 13 additions & 0 deletions

File tree

lib/fluent/plugin/parser_csv.rb

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,11 @@ def configure(conf)
4747

4848
def parse(text, &block)
4949
values = CSV.parse_line(text, col_sep: @delimiter)
50+
unless values
51+
yield nil, nil
52+
return
53+
end
54+
5055
r = Hash[@keys.zip(values)]
5156
time, record = convert_values(parse_time(r), r)
5257
yield time, record

test/plugin/test_parser_csv.rb

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,14 @@ def test_parse_with_option_delimiter(param)
101101
end
102102
end
103103

104+
def test_parse_empty_line
105+
d = create_driver('keys' => '["k1","k2","k3"]')
106+
d.instance.parse("") do |time, record|
107+
assert_nil time
108+
assert_nil record
109+
end
110+
end
111+
104112
sub_test_case 'parser' do
105113
data('normal' => 'normal',
106114
'fast' => 'fast')

0 commit comments

Comments
 (0)