feat(llmobs): allow span processor to return None to omit spans #13739
+86
−10
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow users to omit any traces they want by returning none in the user_span_processor method.
Motivation:
I am trying to prevent some auto traces from cluttering my Datadog observability dashboards. These traces create noise and make it harder to focus on the more critical traces. My LLM observability overview is filled with what is clustered as empty input which is incorrect.
e.g. all embedding traces are are just a spammer.
To do that, I am using the new span_processor, and the new way to omit specific spans, will be to return null by the relevant span processor.
I have added a test for that.
The only risk is that there is no indication that span was omitted but I was afraid that a debug log would be too spammy. But please let me know if that will help.
Also there are some telemetric collected about sent span which might also need to be changed, will be great to hear your thoughts
Checklist
Reviewer Checklist