-
Notifications
You must be signed in to change notification settings - Fork 229
feat: Add AG2 instrumentation integration #1849
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
faridun-ag2
wants to merge
7
commits into
pydantic:main
Choose a base branch
from
faridun-ag2:feat/ag2-integration
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
3ac3660
Add AG2 instrumentation integration, docs, tests, and quickstart
faridun-ag2 b4c4a32
Merge branch 'main' into feat/ag2-integration
faridun-ag2 9a57718
Address PR review: eager patching, snapshot tests, positional args, r…
faridun-ag2 6dc26d9
Merge branch 'main' into feat/ag2-integration
faridun-ag2 e264cd7
Fix README hardcoded paths and clarify agent parameter scoping
faridun-ag2 d6f6306
Fix CI lint: add ag2 dev dependency, remove trailing whitespace
faridun-ag2 0fd5284
Fix suppress_instrumentation placement in _wrap_response_process
faridun-ag2 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| --- | ||
| title: AG2 (Multi-Agent Framework) | ||
| description: Instrument AG2 multi-agent conversations with Logfire spans for conversation flow, agent turns, and tool execution. | ||
| integration: logfire | ||
| --- | ||
|
|
||
| [AG2](https://docs.ag2.ai) supports multi-agent orchestration patterns such as two-agent chat and group chat. | ||
| Use [`logfire.instrument_ag2()`][logfire.Logfire.instrument_ag2] to trace conversation lifecycle events, agent turns, and tool execution. | ||
|
|
||
| ## What gets traced | ||
|
|
||
| `logfire.instrument_ag2()` creates spans for: | ||
|
|
||
| - **Conversation lifecycle** (`AG2 conversation`) | ||
| - **Group chat orchestration** (`AG2 group chat run`, `AG2 group chat round`) | ||
| - **Agent turns** (`AG2 agent turn`) | ||
| - **Tool execution** (`AG2 tool execution`) | ||
|
|
||
| LLM provider request spans are still produced by provider-specific integrations (for example [`instrument_openai()`][logfire.Logfire.instrument_openai]), so AG2 spans stay focused on orchestration. | ||
|
|
||
| ## Install | ||
|
|
||
| ```bash | ||
| pip install logfire "ag2[openai]>=0.11.4,<1.0" | ||
| ``` | ||
|
|
||
| ## Basic usage | ||
|
|
||
| ```python skip-run="true" skip-reason="external-connection" | ||
| import os | ||
|
|
||
| from autogen import AssistantAgent, GroupChat, GroupChatManager, LLMConfig, UserProxyAgent | ||
|
|
||
| import logfire | ||
|
|
||
|
|
||
| llm_config = LLMConfig( | ||
| { | ||
| 'model': 'gpt-4o-mini', | ||
| 'api_key': os.getenv('OPENAI_API_KEY'), | ||
| 'api_type': 'openai', | ||
| } | ||
| ) | ||
|
|
||
|
|
||
| def is_termination(msg: dict[str, object]) -> bool: | ||
| content = msg.get('content', '') or '' | ||
| return isinstance(content, str) and 'TERMINATE' in content | ||
|
|
||
|
|
||
| proxy = UserProxyAgent( | ||
| name='user_proxy', | ||
| human_input_mode='NEVER', | ||
| max_consecutive_auto_reply=10, | ||
| code_execution_config=False, | ||
| is_termination_msg=is_termination, | ||
| ) | ||
| research = AssistantAgent(name='research_agent', system_message='Find relevant facts.', llm_config=llm_config) | ||
| analyst = AssistantAgent( | ||
| name='analyst_agent', | ||
| system_message='Summarize the findings and say TERMINATE when done.', | ||
| llm_config=llm_config, | ||
| ) | ||
|
|
||
|
|
||
| @proxy.register_for_execution() | ||
| @research.register_for_llm(description='Search for information') | ||
| def search_knowledge(query: str) -> str: | ||
| return f'Results for {query}' | ||
|
|
||
|
|
||
| group_chat = GroupChat(agents=[proxy, research, analyst], messages=[], max_round=10) | ||
| manager = GroupChatManager(groupchat=group_chat, llm_config=llm_config, is_termination_msg=is_termination) | ||
|
|
||
| logfire.configure(send_to_logfire=False) | ||
| logfire.instrument_ag2(record_content=False) | ||
|
|
||
| # Optional: also instrument OpenAI request spans | ||
| logfire.instrument_openai() | ||
|
|
||
| proxy.run(manager, message='What is AG2?').process() | ||
| ``` | ||
|
|
||
| ## Configuration | ||
|
|
||
| `logfire.instrument_ag2()` supports: | ||
|
|
||
| - `agent`: instrument a specific AG2 agent instance (or iterable of instances); if omitted, instrument globally. | ||
| - `record_content`: include message/tool payload content in span attributes. Defaults to `False`. | ||
| - `suppress_other_instrumentation`: suppress other OTEL instrumentation while AG2 conversation processing runs. | ||
|
|
||
| ## Related docs | ||
|
|
||
| - [Logfire concepts](https://logfire.pydantic.dev/docs/concepts/) | ||
| - [AG2 documentation](https://docs.ag2.ai) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # AG2 + Logfire quickstart (real OpenAI calls) | ||
|
|
||
| This example runs a real AG2 multi-agent conversation and emits Logfire spans. | ||
|
|
||
| ## Requirements | ||
|
|
||
| - Python environment with this repository installed | ||
| - `ag2[openai]` package | ||
| - `OPENAI_API_KEY` | ||
| - Optional: `LOGFIRE_TOKEN` (if you want to send traces to cloud) | ||
|
|
||
| ## Run | ||
|
|
||
| ```bash | ||
| cd /Users/faridunm/Documents/WORK/AG2/Opensource/logfire | ||
| /Users/faridunm/.pyenv/versions/3.13.12/bin/python examples/python/ag2-openai-quickstart/main.py | ||
| ``` | ||
|
|
||
| Optional arguments: | ||
|
|
||
| ```bash | ||
| /Users/faridunm/.pyenv/versions/3.13.12/bin/python examples/python/ag2-openai-quickstart/main.py --model gpt-4o-mini --question "Plan a 2-day trip to Samarkand" --max-round 6 | ||
| ``` | ||
|
|
||
| ## Environment variables | ||
|
|
||
| ```bash | ||
| export OPENAI_API_KEY="<your-openai-key>" | ||
| # Optional cloud export: | ||
| export LOGFIRE_TOKEN="<your-logfire-token>" | ||
| ``` | ||
|
|
||
| If `LOGFIRE_TOKEN` is not set, spans are still created locally with `send_to_logfire=False`. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import os | ||
|
|
||
| from autogen import AssistantAgent, GroupChat, GroupChatManager, LLMConfig, UserProxyAgent | ||
|
|
||
| import logfire | ||
|
|
||
|
|
||
| def parse_args() -> argparse.Namespace: | ||
| """Parse CLI arguments for the AG2 quickstart runner.""" | ||
| parser = argparse.ArgumentParser(description='AG2 + Logfire quickstart with real OpenAI calls') | ||
| parser.add_argument( | ||
| '--model', | ||
| default='gpt-4o-mini', | ||
| help='OpenAI model name (default: gpt-4o-mini)', | ||
| ) | ||
| parser.add_argument( | ||
| '--question', | ||
| default='Briefly explain what AG2 is and why multi-agent orchestration helps in practice.', | ||
| help='Task to run through AG2 group chat', | ||
| ) | ||
| parser.add_argument( | ||
| '--max-round', | ||
| type=int, | ||
| default=6, | ||
| help='Maximum group-chat rounds (default: 6)', | ||
| ) | ||
| return parser.parse_args() | ||
|
|
||
|
|
||
| def is_termination(msg: dict[str, object]) -> bool: | ||
| """Return True when a message indicates the AG2 chat should terminate.""" | ||
| content = msg.get('content', '') or '' | ||
| return isinstance(content, str) and 'TERMINATE' in content | ||
|
|
||
|
|
||
| def _extract_final_message(messages: list[dict[str, object]]) -> str | None: | ||
| for message in reversed(messages): | ||
| content = message.get('content') | ||
| if not isinstance(content, str) or not content.strip(): | ||
| continue | ||
|
|
||
| role = message.get('role') | ||
| if role == 'assistant' or 'TERMINATE' in content: | ||
| return content | ||
|
|
||
| return None | ||
|
|
||
|
|
||
| def main() -> None: | ||
| """Run a real AG2 group chat with OpenAI and emit Logfire spans.""" | ||
| args = parse_args() | ||
|
|
||
| openai_api_key = os.getenv('OPENAI_API_KEY') | ||
| if not openai_api_key: | ||
| raise SystemExit('OPENAI_API_KEY is required to run real OpenAI calls.') | ||
|
|
||
| llm_config = LLMConfig( | ||
| { | ||
| 'model': args.model, | ||
| 'api_key': openai_api_key, | ||
| 'api_type': 'openai', | ||
| } | ||
| ) | ||
|
|
||
| send_to_logfire = bool(os.getenv('LOGFIRE_TOKEN')) | ||
| logfire.configure(send_to_logfire=send_to_logfire) | ||
|
|
||
| proxy = UserProxyAgent( | ||
| name='user_proxy', | ||
| human_input_mode='NEVER', | ||
| max_consecutive_auto_reply=10, | ||
| code_execution_config=False, | ||
| is_termination_msg=is_termination, | ||
| ) | ||
| researcher = AssistantAgent( | ||
| name='researcher', | ||
| system_message='You gather short factual context. Keep it compact.', | ||
| llm_config=llm_config, | ||
| ) | ||
| summarizer = AssistantAgent( | ||
| name='summarizer', | ||
| system_message='You summarize the answer clearly and finish with TERMINATE.', | ||
| llm_config=llm_config, | ||
| ) | ||
|
|
||
| @proxy.register_for_execution() | ||
| @researcher.register_for_llm(description='Lookup short synthetic facts') | ||
| def quick_lookup(topic: str) -> str: | ||
| return f'Quick lookup result about: {topic}' | ||
|
|
||
| group_chat = GroupChat(agents=[proxy, researcher, summarizer], messages=[], max_round=args.max_round) | ||
| manager = GroupChatManager(groupchat=group_chat, llm_config=llm_config, is_termination_msg=is_termination) | ||
|
|
||
| with logfire.instrument_ag2(record_content=False): | ||
| # Optional but recommended for provider-level spans. | ||
| logfire.instrument_openai() | ||
| print(f'Running AG2 chat with model={args.model!r} ...') | ||
| response = proxy.run(manager, message=args.question) | ||
| final_output = response.process() | ||
|
|
||
| if final_output is None: | ||
| typed_messages: list[dict[str, object]] = list(manager.groupchat.messages) | ||
| final_output = _extract_final_message(typed_messages) | ||
|
|
||
| print('\n=== Final output ===\n') | ||
| print(final_output if final_output is not None else '(No final content returned by AG2 response.process())') | ||
| print('\nDone. Open Logfire to inspect AG2 conversation, rounds, turns, and tool spans.') | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.