Skip to content

feat(providers): add HoneyHiveProvider - #419

Open
sanjeed5 wants to merge 2 commits into
strands-agents:mainfrom
sanjeed5:feat/honeyhive-provider
Open

sanjeed5 wants to merge 2 commits into
strands-agents:mainfrom
sanjeed5:feat/honeyhive-provider

Conversation

@sanjeed5

@sanjeed5 sanjeed5 commented Oct 9, 2026

Copy link
Copy Markdown

Description

Adds HoneyHiveProvider, a TraceProvider that fetches a Strands agent session from HoneyHive and converts it to the evals trace types, so evaluators can score stored sessions without re-running the agent.

  • invoke_agent spans become AgentInvocationSpan, model events become InferenceSpan, and execute_tool spans become ToolExecutionSpan. Event-loop-cycle spans are dropped. Events that fail to convert are logged and skipped.
  • Works with sessions from Strands Python (older and current SDK versions) and Strands TypeScript.
  • Fetches events from POST /v1/events/search, paginating until the reported count. ReadTimeout is retried.
  • Needs a HoneyHive project API key with read access. A 401/403 raises ProviderError with a clear message.
  • Configured with api_key / api_url, or the HH_API_KEY / HH_API_URL environment variables.
  • Uses httpx directly rather than the HoneyHive SDK, so evals installs don't pull in the HoneyHive tracer and its OpenTelemetry dependencies. The optional honeyhive extra only declares httpx, which strands-agents already depends on.

Related Issues

Closes #418

Documentation PR

N/A. The provider README section is updated in this PR.

Type of Change

New feature

Testing

  • Unit tests use three recorded HoneyHive sessions (Strands Python older SDK, Strands Python current SDK, Strands TypeScript) and a mock HTTP transport. No network calls.

  • They cover conversion, tool call and result pairing, tool span shapes, parallel tool results, string tool input, skipping events that fail to convert, pagination, retries, rejected API keys, and a missing session (17 tests).

  • ruff check, ruff format --check, mypy -p src, and the full unit suite (2233 tests) pass.

  • Also checked against live HoneyHive sessions with Experiment(...).run_evaluations(provider.as_task()) and LLM-judge evaluators.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Add a TraceProvider that fetches a Strands agent session from HoneyHive
and converts it to the evals trace types, so evaluators can score stored
sessions without re-running the agent. It reads sessions from Strands
Python and Strands TypeScript agents through the HoneyHive events search
API, and needs a project API key with read access.
@sanjeed5
sanjeed5 requested a review from a team as a code owner October 9, 2026 00:28
@sanjeed5
sanjeed5 requested a review from mkmeral October 9, 2026 00:28
@github-actions github-actions Bot added area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL enhancement New feature or request labels Oct 9, 2026
Resolve the pyproject.toml conflict in the hatch-test and
hatch-static-analysis feature lists by keeping both the honeyhive
and stickler extras.

Claude-Session: https://claude.ai/code/session_01KtJZ5cv5P3EbGLKEd1TibZ

This branch is waiting to be deployed

1 waiting deployment
manual-approval — b7a84dec Waiting Oct 10, 2026 by sanjeed5 via Trigger Strands Review #606
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-tracing Trace/session ingestion: providers, session mappers, extractors, telemetry/OTEL enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a HoneyHive trace provider

2 participants