[exporter/signalfx] Add stateful deduplication for dimension updates #46042
+392
−32
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds hash-based stateful deduplication for dimension property updates to prevent sending duplicate updates to SignalFx backend. This addresses the issue where k8scluster receiver periodically resends entity events every `metadata_collection_interval` (default 5 minutes), causing redundant dimension property updates. For now, behind feature gate `exporter.signalfx.dimensionStatefulDeduplication` (Alpha stage). A follow-up PR will add adaptive TTL for the hash cache to manage memory usage over time and drop keys for dimensions that aren't active anymore.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-up to #45595.
This change adds hash-based, stateful deduplication for dimension property updates to prevent duplicate API calls.
It addresses the issue where the
k8s_clusterreceiver periodically resends entity events everymetadata_collection_interval(default: 5 minutes), which can trigger redundant dimension property updates. Even if periodic events are disabled withmetadata_collection_interval: 0on thek8s_clusterreceiver side, identical entity events can still be sent because the entity payload is only a subset of the full Kubernetes object. Other parts of the Kubernetes object may change in ways that do not affect the captured entity payload, but still cause the event to be re-emitted.For now, this is behind the feature gate
exporter.signalfx.dimensionStatefulDeduplicationand is disabled by default.A follow-up PR will add an adaptive TTL for the hash cache to manage memory usage over time and evict keys for dimensions that are no longer active.