Repository navigation
[FEATURE] Include full metadata in hook events #4015
Description
SDK Language
Python
Problem Statement
I want to monitor the results of Bedrock Guardrails in Hooks when I'm using Bedrock Converse API as a model, so I can change invocation/request state with information about which guardrail was triggered and/or whether non-blocking guardrails were triggered
Proposed Solution
Currently, when us the only way to know which guardrail is triggered is to set a callback and listen for a metadata event, which contains the full metadata from the agent invocation. For example:
{'event': {'metadata': {'usage': {'inputTokens': 67, 'outputTokens': 386, 'totalTokens': 453}, 'metrics': {'latencyMs': 6668}, 'trace': {'guardrail': {'inputAssessment': {'<id>': {'invocationMetrics': {'guardrailProcessingLatency': 132, 'usage': {'topicPolicyUnits': 0, 'contentPolicyUnits': 0, 'wordPolicyUnits': 0, 'sensitiveInformationPolicyUnits': 0, 'sensitiveInformationPolicyFreeUnits': 0, 'contextualGroundingPolicyUnits': 0}, 'guardrailCoverage': {'textCharacters': {'guarded': 77, 'total': 206}}}, 'appliedGuardrailDetails': {'guardrailId': '<id>', 'guardrailVersion': '4', 'guardrailArn': '<arn>', 'guardrailOrigin': ['REQUEST'], 'guardrailOwnership': 'SELF'}}}, 'outputAssessments': {'<id>': [{'topicPolicy': {'topics': [{'name': 'Personalised Financial Advice', 'type': 'DENY', 'action': 'NONE', 'detected': True}, {'name': 'Negative comments about government', 'type': 'DENY', 'action': 'NONE', 'detected': False}, {'name': 'Encouraging Skirting or Ignoring Laws or Regulations', 'type': 'DENY', 'action': 'NONE', 'detected': False}]}, 'contentPolicy': {'filters': [{'type': 'SEXUAL', 'confidence': 'NONE', 'filterStrength': 'HIGH', 'action': 'NONE', 'detected': False}, {'type': 'VIOLENCE', 'confidence': 'NONE', 'filterStrength': 'HIGH', 'action': 'NONE', 'detected': False}, {'type': 'HATE', 'confidence': 'NONE', 'filterStrength': 'HIGH', 'action': 'NONE', 'detected': False}, {'type': 'INSULTS', 'confidence': 'NONE', 'filterStrength': 'HIGH', 'action': 'NONE', 'detected': False}, {'type': 'MISCONDUCT', 'confidence': 'NONE', 'filterStrength': 'HIGH', 'action': 'NONE', 'detected': False}]}, 'sensitiveInformationPolicy': {'piiEntities': [{'match': '', 'type': 'AGE', 'action': 'NONE', 'detected': False}]}, 'invocationMetrics': {'guardrailProcessingLatency': 475, 'usage': {'topicPolicyUnits': 3, 'contentPolicyUnits': 3, 'wordPolicyUnits': 0, 'sensitiveInformationPolicyUnits': 3, 'sensitiveInformationPolicyFreeUnits': 0, 'contextualGroundingPolicyUnits': 0}, 'guardrailCoverage': {'textCharacters': {'guarded': 1074, 'total': 1074}}}, 'appliedGuardrailDetails': {'guardrailId': '<id>', 'guardrailVersion': '4', 'guardrailArn': '<arn>', 'guardrailOrigin': ['REQUEST'], 'guardrailOwnership': 'SELF'}}]}, 'actionReason': 'No action.'}}}}}
Which is triggered from:
harness-sdk/strands-py/src/strands/models/bedrock.py
Lines 1625 to 1634 in 45bcea8
At the point of triggering hooks the trace metadata is stripped so hooks do not have this information:
as just usage and metrics has been extracted leaving traces behind.It is then not available in AfterModelCallEvent.ModelStopResponse message attribute, despite the type signature being suitable for arbitrary data:
harness-sdk/strands-py/src/strands/types/content.py
Lines 215 to 229 in 45bcea8
Use Case
This would be an approach to allow changing invocation state, or other activities, in hooks working with the cleaner interface a hook gives compared to events and easier ability to interact with the agent
Alternatives Solutions
The approach we're using to work around this is using global agent state or global variables, that we need to remember to reset and putting a callback on strands e.g:
def make_guardrail_callback(agent: Agent):
def handler(**kwargs: Any) -> None:
chunk = kwargs.get("event")
if not chunk:
return
guardrail = chunk.get("metadata", {}).get("trace", {}).get("guardrail")
if not guardrail:
return
assessments = extract_guardrail_assessments(guardrail)
if not assessments:
return
agent.state.set("guardrail_triggered", True)
existing = agent.state.get("assessments") or []
existing.extend(assessments)
agent.state.set("assessments", existing)
return handler
Additional Context
No response
Activity
lifelmy commentedon Aug 31, 2026
I opened a draft PR that preserves the
traceportion of the model metadata event so it reaches the assistant message metadata and is available inAfterModelCallEvent.ModelStopResponse.message: #4086.Approach: capture
traceinprocess_streamand attach it to the assistant message metadata (also added atracefield toMessageMetadata). TheModelStopReasontuple shape is left unchanged to avoid touching its many consumers, and the key is only added when the provider actually emits a trace. Added regression tests asserting the trace surfaces on the assistant message and that notracekey is added when absent.I use a coding assistant to help implement, and I review and take responsibility for the final change. Happy to adjust the shape/placement if maintainers prefer a different surface.