in_forward: enable skip_invalid_event by default#5003
Merged
Conversation
Signed-off-by: Shizuo Fujita <fujita@clear-code.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Fixes #
What this PR does / why we need it:
This PR will enable
skip_invalid_eventinin_forwardby default.If broken data is received for any reason, it may cause the following error.
When enable
skip_invalid_event, the broken data are just skipped.And, when
require_ack_responsewas configured totruein out_forward,error_class=TypeError error="can't convert String into an exact number"error by broken data.error_class=Fluent::Plugin::ForwardOutput::NoNodesAvailable error="no nodes are available"error because in_forward doesn't send the ack response.1.in_forward and out_forward will repeat errors and retries indefinitely.
If
skip_invalid_eventwas configured totruein in_forward,in_forward doesn't cause the error by broken data,
in_forward and out_forward don't repeat errors and retries.
So, Fluentd will be more stable by enabled
skip_invalid_event.About performance degradation
It shows the processing time of
in_forwardwhen receiving 1 GB of data.skip_invalid_event falseskip_invalid_event trueEven when skip_invalid_event was enabled, I think the performance degradation was minimal.
Docs Changes:
fluent/fluentd-docs-gitbook#587
Release Note:
Same as the title.