Skip to content

Commit 25c5ddf

Browse files
committed
administration: buffering-and-storage: fix chunks up indication
Signed-off-by: lecaros <[email protected]>
1 parent 130498e commit 25c5ddf

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

administration/buffering-and-storage.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ Enabling filesystem buffering changes the behavior of the engine. Upon chunk cre
103103

104104
Fluent Bit controls the number of chunks that are `up` in memory by using the filesystem buffering mechanism to deal with high memory usage and backpressure.
105105

106-
By default, the engine allows a total of 128 chunks `up` in memory in total, considering all chunks. This value is controlled by the service property `storage.max_chunks_up`. The active chunks that are `up` are ready for delivery and are still receiving records. Any other remaining chunk is in a `down` state, which means that it's only in the filesystem and won't be `up` in memory unless it's ready to be delivered. Chunks are never much larger than 2&nbsp;MB, so with the default `storage.max_chunks_up` value of 128, each input is limited to roughly 256&nbsp;MB of memory.
106+
By default, the engine allows a total of 128 chunks `up` in memory in total, considering all chunks. This value is controlled by the service property `storage.max_chunks_up`. The active chunks that are `up` are either ready for delivery (marked busy and locked), or are still receiving records. Any other remaining chunk is in a `down` state, which means that it's only in the filesystem and won't be `up` in memory unless it's ready to be delivered. Chunks are never much larger than 2&nbsp;MB, so with the default `storage.max_chunks_up` value of 128, each input is limited to roughly 256&nbsp;MB of memory.
107107

108108
If the input plugin has enabled `storage.type` as `filesystem`, when reaching the `storage.max_chunks_up` threshold, instead of the plugin being paused, all new data will go to chunks that are `down` in the filesystem. This lets you control memory usage by the service and also provides a guarantee that the service won't lose any data. By default, the enforcement of the `storage.max_chunks_up` limit is best-effort. Fluent Bit can only append new data to chunks that are `up`. When the limit is reached chunks will be temporarily brought `up` in memory to ingest new data, and then put to a `down` state afterwards. In general, Fluent Bit works to keep the total number of `up` chunks at or under `storage.max_chunks_up`.
109109

0 commit comments

Comments
 (0)