Skip to content

Commit c7044c6

Browse files
committed
Remove wrong description about path of file buffer plugin
Although it seems based on v0.12, it's not applicable for v0.14 or later. We should rewrite it. Signed-off-by: Takuro Ashie <[email protected]>
1 parent 00e87fe commit c7044c6

File tree

2 files changed

+4
-85
lines changed

2 files changed

+4
-85
lines changed

buffer/file.md

Lines changed: 4 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -9,49 +9,11 @@ The `file` buffer plugin provides a persistent buffer implementation. It uses fi
99

1010
### `path`
1111

12-
| type | required | default | version |
13-
| :--- | :---: | :--- | :--- |
14-
| string || | 0.9.0 |
15-
16-
The path where buffer chunks are stored. The '\*' is replaced with random characters. This parameter is required.
17-
18-
This parameter must be unique to avoid the race condition problem. For example, you cannot use a fixed path parameter in `fluent-plugin-forest`. `${tag}` or similar placeholder is needed. Of course, this parameter must also be unique between fluentd instances.
19-
20-
In addition, `path` should not be another `path` prefix. For example, the following configuration does not work well. `/var/log/fluent/foo` resumes `/var/log/fluent/foo.bar`'s buffer files during start phase and it causes `No such file or directory` in `/var/log/fluent/foo.bar` side.
21-
22-
```text
23-
<match pattern1>
24-
<buffer>
25-
@type file
26-
path /var/log/fluent/foo
27-
</buffer>
28-
</match>
29-
30-
<match pattern2>
31-
<buffer>
32-
@type file
33-
path /var/log/fluent/foo.bar
34-
</buffer>
35-
</match>
36-
```
37-
38-
Here is the correct version to avoid the prefix problem:
12+
| type | default | version |
13+
| :--- | :--- | :--- |
14+
| string | nil | TBD |
3915

40-
```text
41-
<match pattern1>
42-
<buffer>
43-
@type file
44-
path /var/log/fluent/foo.baz
45-
</buffer>
46-
</match>
47-
48-
<match pattern2>
49-
<buffer>
50-
@type file
51-
path /var/log/fluent/foo.bar
52-
</buffer>
53-
</match>
54-
```
16+
The path where buffer chunks are stored.
5517

5618
Please make sure that you have **enough space in the path directory**. Running out of disk space is a problem frequently reported by users.
5719

@@ -88,10 +50,6 @@ This parameter is useful when `.log` is not fit for your environment. See also [
8850

8951
## Tips
9052

91-
### Multi-Process Environment
92-
93-
If you use this plugin under the multi-process environment, you need to use `@id`/`root_dir` parameters instead of fixed `path` parameter. See [Multi Process Workers](../deployment/multi-process-workers.md#root_dir-id-parameter) article.
94-
9553
## Limitation
9654

9755
Caution: `file` buffer implementation depends on the characteristics of the local file system. Don't use `file` buffer on remote file systems e.g. NFS, GlusterFS, HDFS, etc. We observed major data loss by using the remote file system.

deployment/multi-process-workers.md

Lines changed: 0 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -131,45 +131,6 @@ As of Fluentd v1.4.0, `<worker N-M>` syntax has been introduced:
131131

132132
With this directive, you can specify multiple workers per worker directive.
133133

134-
### `root_dir/@id` parameter
135-
136-
These parameters must be specified when you use the file buffer.
137-
138-
With multi-process workers, you cannot use the fixed `path` configuration for file buffer because it conflicts buffer file path between processes.
139-
140-
```text
141-
<system>
142-
workers 2
143-
</system>
144-
145-
<match pattern>
146-
@type forward
147-
<buffer>
148-
@type file
149-
path /var/log/fluentd/forward # This is not allowed
150-
</buffer>
151-
</match>
152-
```
153-
154-
Instead of a fixed configuration, fluentd provides the dynamic buffer path based on `root_dir` and `@id` parameters. The stored path is `${root_dir}/worker${worker index}/${plugin @id}/buffer` directory.
155-
156-
```text
157-
<system>
158-
workers 2
159-
root_dir /var/log/fluentd
160-
</system>
161-
162-
<match pattern>
163-
@type forward
164-
@id out_fwd
165-
<buffer>
166-
@type file
167-
</buffer>
168-
</match>
169-
```
170-
171-
With this configuration, `forward` output buffer files are stored into `/var/log/fluentd/worker0/out_fwd/buffer` and `/var/log/fluentd/worker1/out_fwd/buffer` directories.
172-
173134
## Operation
174135

175136
Each worker consumes memory and disk space separately. Take care while configuring buffer spaces and monitoring memory/disk consumption.

0 commit comments

Comments
 (0)