Commit 0547ff9
authored
parser_csv: skip empty or unparseable lines (#5355)
**Which issue(s) this PR fixes**:
Fixes #
**What this PR does / why we need it**:
This PR fixes an issue in `parser_csv` where processing an empty line
causes a `TypeError`.
When `parser_csv` receives an empty line, `CSV.parse_line` returns
`nil`.
Currently, passing this `nil` value to `Array#zip` throws a `TypeError:
wrong argument type NilClass (must respond to :each)`.
### Reproduce
config:
```
<source>
@type tail
path "#{File.expand_path '~/tmp/fluentd/test.csv'}"
tag test.csv
read_from_head true
<parse>
@type csv
keys k1,k2,k3
</parse>
</source>
<match test.csv>
@type stdout
</match>
```
Prepare input data:
```
$ echo "A,B,C" > ~/tmp/fluentd/test.csv
$ echo "" >> ~/tmp/fluentd/test.csv
$ echo "D,E,F" >> ~/tmp/fluentd/test.csv
```
Result of before changing:
```
2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv
2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found
2026-05-09 17:23:14 +0900 [warn]: #0 this parameter is highly recommended to save the position to resume tailing.
2026-05-09 17:23:14 +0900 [info]: #0 starting fluentd worker pid=41671 ppid=41646 worker=0
2026-05-09 17:23:14 +0900 [info]: #0 following tail of /home/watson/tmp/fluentd/test.csv
2026-05-09 17:23:14 +0900 [warn]: #0 invalid line found file="/home/watson/tmp/fluentd/test.csv" line="" error="wrong argument type NilClass (must respond to :each)"
2026-05-09 17:23:14.692710744 +0900 test.csv: {"k1":"A","k2":"B","k3":"C"}
2026-05-09 17:23:14.692801343 +0900 test.csv: {"k1":"D","k2":"E","k3":"F"}
2026-05-09 17:23:14 +0900 [info]: #0 fluentd worker is now running worker=0
```
**Docs Changes**:
N/A
**Release Note**:
* parser_csv: skip empty or unparseable lines
Signed-off-by: Shizuo Fujita <fujita@clear-code.com>1 parent 4b7b6b2 commit 0547ff9
2 files changed
Lines changed: 13 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
50 | 55 | | |
51 | 56 | | |
52 | 57 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
101 | 101 | | |
102 | 102 | | |
103 | 103 | | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
104 | 112 | | |
105 | 113 | | |
106 | 114 | | |
| |||
0 commit comments