Skip to content

Backport(v1.19): parser_csv: skip empty or unparseable lines (#5355)#5359

Merged
Watson1978 merged 1 commit into
v1.19from
backport-to-v1.19/pr5355
May 17, 2026
Merged

Backport(v1.19): parser_csv: skip empty or unparseable lines (#5355)#5359
Watson1978 merged 1 commit into
v1.19from
backport-to-v1.19/pr5355

Conversation

@github-actions

Copy link
Copy Markdown

Which issue(s) this PR fixes:
Backport #5355
Fixes #

What this PR does / why we need it:
This PR fixes an issue in parser_csv where processing an empty line causes a TypeError.

When parser_csv receives an empty line, CSV.parse_line returns nil.
Currently, passing this nil value to Array#zip throws a TypeError: wrong argument type NilClass (must respond to :each).

Reproduce

config:

<source>
  @type tail
  path "#{File.expand_path '~/tmp/fluentd/test.csv'}"
  tag test.csv
  read_from_head true
  <parse>
    @type csv
    keys k1,k2,k3
  </parse>
</source>

<match test.csv>
  @type stdout
</match>

Prepare input data:

$ echo "A,B,C"  > ~/tmp/fluentd/test.csv
$ echo ""      >> ~/tmp/fluentd/test.csv
$ echo "D,E,F" >> ~/tmp/fluentd/test.csv

Result of before changing:

2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv
2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found 
2026-05-09 17:23:14 +0900 [warn]: #0 this parameter is highly recommended to save the position to resume tailing.
2026-05-09 17:23:14 +0900 [info]: #0 starting fluentd worker pid=41671 ppid=41646 worker=0
2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv
2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found file="/home/watson/tmp/fluentd/test.csv" line="" error="wrong argument type NilClass (must respond to :each)"
2026-05-09 17:23:14.692710744 +0900 test.csv: {"k1":"A","k2":"B","k3":"C"}
2026-05-09 17:23:14.692801343 +0900 test.csv: {"k1":"D","k2":"E","k3":"F"}
2026-05-09 17:23:14 +0900 [info]: #0 fluentd worker is now running worker=0

Docs Changes:
N/A

Release Note:

  • parser_csv: skip empty or unparseable lines

**Which issue(s) this PR fixes**:
Fixes #

**What this PR does / why we need it**:
This PR fixes an issue in `parser_csv` where processing an empty line
causes a `TypeError`.

When `parser_csv` receives an empty line, `CSV.parse_line` returns
`nil`.
Currently, passing this `nil` value to `Array#zip` throws a `TypeError:
wrong argument type NilClass (must respond to :each)`.

### Reproduce
config:
```
<source>
  @type tail
  path "#{File.expand_path '~/tmp/fluentd/test.csv'}"
  tag test.csv
  read_from_head true
  <parse>
    @type csv
    keys k1,k2,k3
  </parse>
</source>

<match test.csv>
  @type stdout
</match>
```

Prepare input data:
```
$ echo "A,B,C"  > ~/tmp/fluentd/test.csv
$ echo ""      >> ~/tmp/fluentd/test.csv
$ echo "D,E,F" >> ~/tmp/fluentd/test.csv
```

Result of before changing:
```
2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv
2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found
2026-05-09 17:23:14 +0900 [warn]: #0 this parameter is highly recommended to save the position to resume tailing.
2026-05-09 17:23:14 +0900 [info]: #0 starting fluentd worker pid=41671 ppid=41646 worker=0
2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv
2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found file="/home/watson/tmp/fluentd/test.csv" line="" error="wrong argument type NilClass (must respond to :each)"
2026-05-09 17:23:14.692710744 +0900 test.csv: {"k1":"A","k2":"B","k3":"C"}
2026-05-09 17:23:14.692801343 +0900 test.csv: {"k1":"D","k2":"E","k3":"F"}
2026-05-09 17:23:14 +0900 [info]: #0 fluentd worker is now running worker=0
```

**Docs Changes**:
N/A

**Release Note**:
* parser_csv: skip empty or unparseable lines

Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@Watson1978 Watson1978 merged commit c16dd7e into v1.19 May 17, 2026
1 check passed
@Watson1978 Watson1978 deleted the backport-to-v1.19/pr5355 branch May 17, 2026 02:29
@Watson1978 Watson1978 added this to the v1.19.3 milestone May 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant