Skip to content

feat(hooks): preserve model trace metadata for hook events (#4015) - #4086

Open
lifelmy wants to merge 154 commits into
strands-agents:mainfrom
lifelmy:agent-tasks/4015
Open

lifelmy wants to merge 154 commits into
strands-agents:mainfrom
lifelmy:agent-tasks/4015

Conversation

@lifelmy

@lifelmy lifelmy commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Description

Closes #4015.

When a model emits a metadata event containing a trace (e.g. Bedrock guardrail assessments returned by the Converse API), the event loop extracted only usage and metrics and discarded the trace. As a result the trace was never attached to the assistant message metadata and was unavailable in AfterModelCallEvent.ModelStopResponse.message, so hooks could not inspect guardrail results without resorting to a callback + global/agent state workaround (as described in the issue).

Changes

  • event_loop/streaming.py: capture the trace portion of the metadata chunk in process_stream and attach it to the constructed assistant message metadata. The ModelStopReason tuple shape is intentionally left unchanged (it has many consumers), so the trace travels on the message rather than by widening the tuple.
  • event_loop/event_loop.py: when populating message["metadata"] (usage/metrics), preserve any trace captured during streaming so it reaches hooks/events/state.
  • types/content.py: add an optional trace field to MessageMetadata.

The trace key is only added when the provider actually emits a trace, so messages without a trace are unchanged.

Testing

  • hatch test tests/strands/event_loop/test_event_loop_metadata.py — 5 passed (2 new regression tests: trace surfaces on the assistant message metadata; no trace key added when absent).
  • hatch test tests/strands/event_loop/test_streaming.py tests/strands/event_loop/test_event_loop.py — 144 passed.
  • hatch test tests/strands/hooks tests/strands/types — 167 passed.
  • ruff check / ruff format --check — clean; mypy on the changed modules — no issues.

Note

I use a coding assistant to help implement, and I review and take responsibility for the final change.

…gents#4015)

The trace portion of the model metadata event (e.g. Bedrock guardrail
assessments) was dropped after usage/metrics extraction, so it never
reached the assistant message metadata and was unavailable in
AfterModelCallEvent.ModelStopResponse.message.

Capture trace in process_stream and thread it onto the assistant message
metadata (via event_loop), and add a trace field to MessageMetadata. The
ModelStopReason tuple shape is unchanged to avoid touching its many
consumers.
@github-actions github-actions Bot added enhancement New feature or request area-hooks Features or requests that might be implementable via hooks area-interventions Related to interventions python Pull requests that update python code labels Aug 31, 2026
@lifelmy
lifelmy marked this pull request as ready for review September 3, 2026 09:27
@lifelmy
lifelmy requested a review from a team as a code owner September 3, 2026 09:27
@lifelmy
lifelmy requested a review from Unshure September 3, 2026 09:27
@lifelmy

lifelmy commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed this branch onto the latest upstream/main (34d0362ef, fix(bidi): rename reconnect to restart (#4143)); new head is ea32ef8fa. The merge completed without conflicts. The upstream commit only touches experimental/bidi, while this PR remains limited to event-loop trace metadata.

Verification after the refresh:

  • .venv/bin/pytest -q tests/strands/event_loop/test_event_loop_metadata.py → 5 passed
  • uv run --extra bidi-all pytest -q tests/strands/experimental/bidi/agent/test_agent.py tests/strands/experimental/bidi/agent/test_loop.py tests/strands/experimental/bidi/agent/test_loop_telemetry.py tests/strands/experimental/bidi/models/test_bedrock.py tests/strands/experimental/bidi/models/test_google.py tests/strands/experimental/bidi/models/test_model.py tests/strands/experimental/bidi/models/test_openai.py → 203 passed
  • git diff --check upstream/main...HEAD → clean

@lifelmy

lifelmy commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed this branch onto the latest upstream/main (24bc5c3d5) after the base moved; the merge completed without conflicts and only incorporated upstream's Node.js 22 baseline update outside the Python feature diff. New head: 39c34894e.

Verification on the refreshed head:

  • uv run --extra bidi-all pytest -q tests/strands/event_loop/test_event_loop_metadata.py tests/strands/event_loop/test_streaming.py → 78 passed
  • git diff --check upstream/main...HEAD → clean

The trace-metadata behavior remains covered on the current base.

@lifelmy

lifelmy commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed this PR onto the latest upstream/main (2acc97487); current head is 0d86b4596. The merge completed without manual conflict edits and preserved the hook-event trace metadata behavior.

Verification from strands-py/:

  • uv run pytest -q tests/strands/event_loop/test_event_loop_metadata.py tests/strands/event_loop/test_streaming.py -> 78 passed
  • focused uv run ruff check -> passed
  • git diff --check upstream/main...HEAD -> passed

GitHub reports the branch as mergeable; the maintainer-controlled Strands review gate remains pending.

@lifelmy

lifelmy commented Sep 4, 2026

Copy link
Copy Markdown
Contributor Author

Refreshed this PR onto the latest upstream/main (f4a3191); new head is f5417f1. The branch was 1 commit behind and merged cleanly without manual conflict edits. Verification from strands-py: 78 focused event-loop tests passed; focused ruff and git diff --check passed.

liumingyao.marvin and others added 30 commits October 10, 2026 03:05
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>
…tasks/4015

Co-authored-by: TRAE CLI <traecli@bytedance.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-hooks Features or requests that might be implementable via hooks area-interventions Related to interventions enhancement New feature or request python Pull requests that update python code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Include full metadata in hook events

1 participant