Skip to content

enhancement(sample transform): Add group tag to discarded event metrics - #26404

Open
ArunPiduguDD wants to merge 23 commits into
arun.pidugu/throttle-discarded-events-tag-group-byfrom
arun.pidugu/sample-discarded-events-tag-group-by
Open

enhancement(sample transform): Add group tag to discarded event metrics#26404
ArunPiduguDD wants to merge 23 commits into
arun.pidugu/throttle-discarded-events-tag-group-byfrom
arun.pidugu/sample-discarded-events-tag-group-by

Conversation

@ArunPiduguDD

@ArunPiduguDD ArunPiduguDD commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Add an opt-in internal_metrics.include_group_tag setting to the sample transform. When enabled, component_discarded_events_total includes the rendered group_by value as a group tag, with None used for the default group.

This PR is stacked on #26402.

References

Vector configuration

transforms:
  sample:
    type: sample
    inputs: [source]
    ratio: 0.5
    group_by: "{{ service }}"
    internal_metrics:
      include_group_tag: true

How did you test this PR?

  • make test FEATURES="transforms-sample" SCOPE="sample"
  • make fmt
  • make check-fmt
  • make check-markdown
  • make check-changelog-fragments
  • make generate-docs
  • cargo clippy --workspace --all-targets --no-default-features --features transforms-sample -- -A clippy::unused_async

The standard Clippy command is blocked by an unrelated existing clippy::unused_async finding in src/test_util/http.rs under Rust 1.96.

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

Contributor Guidelines

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Before pushing, follow our pre-push guidance.
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.

@ArunPiduguDD
ArunPiduguDD requested review from a team as code owners September 16, 2026 19:43
@github-actions github-actions Bot added docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. domain: transforms Anything related to Vector's transform components domain: external docs Anything related to Vector's external, public documentation and removed docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. labels Sep 16, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 16, 2026 •
edited
Loading

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-06T20:30:01.993950Z aaa4987 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Sep 16, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 105b3c1e61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/internal_events/sample.rs Outdated
Comment thread src/transforms/sample/transform.rs Outdated
@ArunPiduguDD
ArunPiduguDD marked this pull request as draft September 17, 2026 03:49
@github-actions github-actions Bot added the docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. label Sep 17, 2026
@ArunPiduguDD
ArunPiduguDD marked this pull request as ready for review September 21, 2026 17:22
@ArunPiduguDD
ArunPiduguDD added this pull request to stack #26449 September 21, 2026 17:22
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Sep 21, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e0a1048dc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/internal_events/sample.rs Outdated
@ArunPiduguDD ArunPiduguDD added type: enhancement A value-adding code change that enhances its existing functionality. domain: config Anything related to configuring Vector domain: metrics Anything related to Vector's metrics events labels Sep 22, 2026 — with ddtool CLI
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Sep 22, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8d821fa26b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/transforms/sample/transform.rs Outdated
ArunPiduguDD and others added 4 commits September 29, 2026 15:33
@ArunPiduguDD @pi-mono
Add sample group tag to discarded events
6fdd0b7 
Allow sample transforms to opt in to grouping discarded-event metrics by rendered group value.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 <noreply@pi.dev>
@ArunPiduguDD @pi-mono
Preserve dropped event instrumentation in sample stack
28143d7 
Apply the parent instrumentation fix and emit the standard dropped-event log for sample metrics.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 <noreply@pi.dev>
@ArunPiduguDD @pi-mono
Restore sample discarded event name
473774f 
Keep the established internal event name and clarify the group tag cardinality tradeoff.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 Sol <noreply@pi.dev>
@ArunPiduguDD @pi-mono
Address sample metric review feedback
beb7e27 
Preserve dropped-event logging and avoid group-key cloning when the tag is disabled.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 Sol <noreply@pi.dev>
@ArunPiduguDD
ArunPiduguDD force-pushed the arun.pidugu/sample-discarded-events-tag-group-by branch from 8d821fa to beb7e27 Compare September 29, 2026 15:50
@ArunPiduguDD ArunPiduguDD changed the title Add sample group tag to discarded event metrics enhancement(sample transform): add group tag to discarded event metrics Sep 29, 2026
ArunPiduguDD and others added 5 commits September 29, 2026 16:57
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
5e02a20 
…o arun.pidugu/sample-discarded-events-tag-group-by
@ArunPiduguDD @pi-mono
Avoid untagged sample group allocation
1453ba2 
Carry the optional group into the internal event and create the fallback only for tagged metrics.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 Sol <noreply@pi.dev>
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
08dcd16 
…o arun.pidugu/sample-discarded-events-tag-group-by
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
9414ee5 
…o arun.pidugu/sample-discarded-events-tag-group-by
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
9562b26 
…o arun.pidugu/sample-discarded-events-tag-group-by
ArunPiduguDD and others added 4 commits September 30, 2026 01:41
@ArunPiduguDD @pi-mono
Use tagged dropped event helper for sample
a1a7823 
Delegate sample discard logging and metrics to the shared helper while constructing the optional group label only when enabled.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 Sol <noreply@pi.dev>
@ArunPiduguDD @pi-mono
Merge shared dropped-event refactor into sample
0a8d0ed 
Bring the parent helper refactor and tests into the sample branch without rewriting published history.

Environment: Datadog workspace

Co-Authored-By: Pi ai-gw-openai/openai/gpt-6-astra <noreply@pi.dev>
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
09e323e 
…o arun.pidugu/sample-discarded-events-tag-group-by
@ArunPiduguDD @pi-mono
Avoid cloning retained sample groups
2649cc6 
Borrow sampling group keys through the decision, move them only into discarded-event metrics, and document the literal None fallback.

Environment: Datadog workspace

Co-Authored-By: Pi ai-gw-openai/openai/gpt-5.6-sol <noreply@pi.dev>
chatgpt-codex-connector[bot]
chatgpt-codex-connector Bot reviewed Oct 1, 2026
View reviewed changes

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2649cc65ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/transforms/sample/transform.rs Outdated
ArunPiduguDD and others added 5 commits October 1, 2026 22:59
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
1dd2a6b 
…o arun.pidugu/sample-discarded-events-tag-group-by
@ArunPiduguDD @pi-mono
Use group-specific dropped event helper
15bf60d 
Pass sample groups through the narrowed shared helper without exposing arbitrary metric labels.

Environment: Datadog workspace

Co-Authored-By: Pi ai-gw-openai/openai/gpt-5.6-sol <noreply@pi.dev>
@ArunPiduguDD
Merge branch 'arun.pidugu/throttle-discarded-events-tag-group-by' int…
9da0cc0 
…o arun.pidugu/sample-discarded-events-tag-group-by
@ArunPiduguDD @pi-mono
Clarify sample group tag documentation
f05f177 
Describe the optional group tag and None fallback directly in the configuration and generated docs.

Environment: Datadog workspace

Co-Authored-By: Pi ai-gw-openai/openai/gpt-5.6-sol <noreply@pi.dev>
@ArunPiduguDD @pi-mono
Restore original sample state handling
a0e52cc 
Keep the group clone opt-in for discarded-event metrics without refactoring sampling state management.

Environment: Datadog workspace

Co-Authored-By: Pi ai-gw-openai/openai/gpt-6-astra <noreply@pi.dev>
@ArunPiduguDD ArunPiduguDD changed the title enhancement(sample transform): add group tag to discarded event metrics enhancement(sample transform): Add group tag to discarded event metrics Oct 2, 2026
@ArunPiduguDD @pi-mono
Merge dropped-event test cleanup into sample
954272e 
Keep the sample branch aligned with the shared-helper cleanup without rewriting published commits.

Environment: Datadog workspace

Co-Authored-By: Pi ai-gw-openai/openai/gpt-6-astra <noreply@pi.dev>
ArunPiduguDD and others added 4 commits October 5, 2026 17:33
@ArunPiduguDD @pi-mono
Pass group metric setting to sample constructors
bd6e657 
Make the internal metric configuration an explicit constructor dependency.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 <noreply@pi.dev>
@ArunPiduguDD @pi-mono
Merge updated throttle parent into sample branch
40b85ce 
Include the OPW Vector baseline while preserving the published stack history.

Environment: Datadog workspace

Co-Authored-By: Pi OpenAI GPT-5.6 <noreply@pi.dev>
@ArunPiduguDD
Merge updated throttle parent into sample branch
ec45e10 
Keep both group-tag changes on the current upstream baseline.

Environment: Datadog workspace
@ArunPiduguDD
Merge throttle checker fix into sample branch
aaa4987 
Keep grouped dropped-event validation consistent across the stack.

Environment: Datadog workspace
@github-actions github-actions Bot removed the docs review on hold The documentation team reviews PRs only after a PR is approved by the COSE team. label Oct 6, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments
@thomasqueirozb thomasqueirozb thomasqueirozb approved these changes
@joepeeples joepeeples joepeeples approved these changes
@dd-sebastien-lb dd-sebastien-lb dd-sebastien-lb approved these changes

Assignees

No one assigned

Labels

domain: config Anything related to configuring Vector domain: external docs Anything related to Vector's external, public documentation domain: metrics Anything related to Vector's metrics events domain: transforms Anything related to Vector's transform components type: enhancement A value-adding code change that enhances its existing functionality.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@ArunPiduguDD @thomasqueirozb @joepeeples @dd-sebastien-lb