Improve OTel GenAI semantic convention compliance for LLM instrumentations #1593
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jimilp7
changed the title
jimilp7
changed the base branch from
main
to
update-llm-instrumentation-semconvs
jimilp7
force-pushed
the
update-llm-instrumentation-semconvs
branch
2 times, most recently
from
abec7c9 to
265e892
Compare
…tions This commit enhances OpenAI and Anthropic instrumentations to better comply with OpenTelemetry GenAI semantic conventions (https://opentelemetry.io/docs/specs/semconv/gen-ai/). Changes: - Set SpanKind=CLIENT for all GenAI API calls (was INTERNAL) - Use OTel-compliant span names in "{operation} {model}" format (e.g., "chat gpt-4o-mini") - Add request parameter attributes: gen_ai.request.max_tokens, temperature, top_p, top_k, stop_sequences, seed, frequency_penalty, presence_penalty - Add gen_ai.tool.definitions when tools are provided in requests - Add server.address and server.port attributes extracted from client base_url - Remove deprecated gen_ai.system attribute from OpenAI instrumentation - Add finish_reasons mapping for OpenAI Responses API (status → finish_reason) Files modified: - main.py: Added _span_kind parameter support to Logfire.span() - llm_provider.py: Set SpanKind.CLIENT, OTel span naming, server attributes - openai.py: Request params extraction, tool definitions, remove deprecated attr - anthropic.py: Request params extraction, tool definitions - semconv.py: Added new semantic convention constants 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <[email protected]>
jimilp7
force-pushed
the
update-llm-instrumentation-semconvs
branch
from
265e892 to
eec1209
Compare
Add LangChain/LangGraph OTel GenAI semantic convention support - Transform LangSmith spans to add OTel GenAI attributes - Add gen_ai.input.messages and gen_ai.output.messages in OTel schema - Extract gen_ai.system_instructions from system messages - Add gen_ai.response.finish_reasons - Add logfire.instrument_langchain() to capture tool definitions - Patches BaseCallbackManager to inject callback handler - Captures gen_ai.tool.definitions (not available via LangSmith OTEL) - Uses _start()/_end() pattern to avoid async context issues
jimilp7
force-pushed
the
update-llm-instrumentation-semconvs
branch
from
5121d6a to
9307923
Compare
jimilp7
force-pushed
the
update-llm-instrumentation-semconvs
branch
from
47a633e to
015babe
Compare
jimilp7
force-pushed
the
update-llm-instrumentation-semconvs
branch
from
54a2096 to
2f34d18
Compare
jimilp7
force-pushed
the
update-llm-instrumentation-semconvs
branch
from
e41de92 to
d35ecc4
Compare
Contributor
|
Please make a new PR that just adds/changes these: SpanKind, Langchain stuff should definitely be completely separated from OpenAI/Anthropic stuff. Changing span names and dropping |
Author
|
@alexmojaki Thanks for review, I am breaking this PR further into 3 distinct prs: I will also close this PR now. |
Author
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR improves OTel GenAI Semantic Convention compliance across LLM instrumentations.
What's New
1. OpenAI & Anthropic Instrumentation Improvements
Enhanced existing instrumentations to better comply with OTel GenAI Semantic Conventions:
INTERNALCLIENT"Chat Completion with {model!r}""chat gpt-4o-mini"gen_ai.system"openai"(deprecated)gen_ai.provider.name"openai"/"anthropic"server.address"api.openai.com"gen_ai.request.*max_tokens,temperature,top_p, etc.gen_ai.tool.definitions2. LangChain/LangGraph Instrumentation (New)
Adds
logfire.instrument_langchain()to capture tool definitions from LangChain agents.Why: LangSmith's OTEL integration does NOT include tool definitions - they're sent via their proprietary API. This instrumentation patches LangChain's callback system to capture them.
Usage:
Captures:
Related