Skip to content

Commit c4bd018

Browse files
authored
Merge pull request #415 from daipom/rewrite-buffer-path-description-fukuda
Add fix to "Rewrite wrong description about path of file buffer plugin"
2 parents c7044c6 + 6c1bfc7 commit c4bd018

File tree

2 files changed

+108
-5
lines changed

2 files changed

+108
-5
lines changed

buffer/file.md

Lines changed: 107 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,81 @@ The `file` buffer plugin provides a persistent buffer implementation. It uses fi
1313
| :--- | :--- | :--- |
1414
| string | nil | TBD |
1515

16-
The path where buffer chunks are stored.
16+
The directory path where buffer chunks are stored. Don't share this directory path with other buffers.
17+
Be sure to specify a unique path for each buffer.
18+
19+
```text
20+
<match pattern>
21+
...
22+
<buffer>
23+
@type file
24+
path /var/log/fluent/buf
25+
</buffer>
26+
</match>
27+
```
28+
29+
This config outputs the buffer chunk files as follows. The file name is `buffer.b{chunk_id}{path_suffix}`.
30+
31+
```text
32+
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
33+
/var/log/fluentd/buf/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta
34+
```
35+
36+
With [multiple workers](../deployment/multi-process-workers.md), a directory is automatically created for each worker.
37+
So there is no need to specify a unique path for each worker.
38+
39+
```text
40+
<system>
41+
workers 2
42+
</system>
43+
44+
...
45+
46+
<match pattern>
47+
...
48+
<buffer>
49+
@type file
50+
path /var/log/fluent/buf
51+
</buffer>
52+
</match>
53+
```
54+
55+
This config outputs the buffer chunk files as follows. The directory `worker{worker_id}` is automatically created.
56+
57+
```text
58+
/var/log/fluentd/buf/worker0/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
59+
/var/log/fluentd/buf/worker0/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta
60+
61+
/var/log/fluentd/buf/worker1/buffer.b5e2a5aca2bcd9818ad6718845ddc456a.log
62+
/var/log/fluentd/buf/worker1/buffer.b5e2a5aca2bcd9818ad6718845ddc456a.log.meta
63+
```
64+
65+
If you specify `root_dir` in [system configuration](../deployment/system-config.md) and [@id](../configuration/plugin-common-parameters.md#id) of the plugin,
66+
then you can omit this parameter.
67+
68+
```text
69+
<system>
70+
root_dir /var/log/fluentd
71+
</system>
72+
73+
...
74+
75+
<match pattern>
76+
@id test_id
77+
...
78+
<buffer>
79+
@type file
80+
</buffer>
81+
</match>
82+
```
83+
84+
This config outputs the buffer chunk files as follows. The directory `{root_dir}/worker{worker_id}/{@id}/buffer` is used for the path.
85+
In this case, the `worker{worker_id}` directory is created even for a single worker.
86+
87+
```text
88+
/var/log/fluentd/worker0/test_id/buffer/buffer.b58eec11d08ca8143b40e4d303510e0bb.log
89+
/var/log/fluentd/worker0/test_id/buffer/buffer.b58eec11d08ca8143b40e4d303510e0bb.log.meta
90+
```
1791

1892
Please make sure that you have **enough space in the path directory**. Running out of disk space is a problem frequently reported by users.
1993

@@ -37,18 +111,47 @@ Changes the suffix of the buffer file.
37111

38112
This parameter is useful when `.log` is not fit for your environment. See also [this issue's comment](https://github.com/fluent/fluentd/issues/2236#issuecomment-514733974).
39113

40-
## Example
114+
## Tips
115+
116+
### Customize a filename of the buffer chunk
117+
118+
You can customize the prefix of filename (`buffer` by default) by adding `.*` to the end of the `path` parameter.
41119

42120
```text
43121
<match pattern>
122+
...
44123
<buffer>
45124
@type file
46-
path /var/log/fluent/myapp.*.buffer
125+
path /var/log/fluent/buf/custom.*
47126
</buffer>
48127
</match>
49128
```
50129

51-
## Tips
130+
This config outputs the buffer chunk files as follows. The prefix `buffer` is changed to `custom`.
131+
132+
```text
133+
/var/log/fluentd/buf/custom.b58eec11d08ca8143b40e4d303510e0bb.log
134+
/var/log/fluentd/buf/custom.b58eec11d08ca8143b40e4d303510e0bb.log.meta
135+
```
136+
137+
You can also customize the entire filename by adding `.*.` to the `path` parameter.
138+
139+
```text
140+
<match pattern>
141+
...
142+
<buffer>
143+
@type file
144+
path /var/log/fluent/buf/custom_prefix.*.custom_suffix
145+
</buffer>
146+
</match>
147+
```
148+
149+
This config outputs the buffer chunk files as follows. In this case, `path_suffix` parameter is not used.
150+
151+
```text
152+
/var/log/fluentd/buf/custom_prefix.b58eec11d08ca8143b40e4d303510e0bb.custom_suffix
153+
/var/log/fluentd/buf/custom_prefix.b58eec11d08ca8143b40e4d303510e0bb.custom_suffix.meta
154+
```
52155

53156
## Limitation
54157

configuration/plugin-common-parameters.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ The `@id` parameter specifies a unique name for the configuration. It is used as
3131
</match>
3232
```
3333

34-
This parameter should be specified for all the plugins to enable `root_dir` and `workers` feature globally.
34+
This parameter should be specified for all the plugins to enable `root_dir` feature globally.
3535

3636
See also: [System Configuration](../deployment/system-config.md)
3737

0 commit comments

Comments
 (0)