[lifelmy]

Description

Closes #143.

Extends TraceProvider with optional session discovery and adds a batch helper so callers can evaluate every session matching a filter in one call, instead of hand-writing the discover → build cases → run boilerplate.

What changed

• SessionFilter (Pydantic model in providers/trace_provider.py): universal start_time / end_time / limit fields plus additional_fields: dict[str, Any] for provider-specific criteria.
• TraceProvider.list_sessions(session_filter=None) -> Iterator[str]: intentionally not abstract. Providers override it only when their backend supports discovery; the default raises NotImplementedError pointing at the known-session_id path (get_evaluation_data). This keeps the bar low for providers that only accept externally-supplied session IDs — matching alternative Add InteractionsEvaluator for multi-agent and more complicated sequence-based evaluations. #1 rejected in the issue.
• strands_evals.batch.evaluate_sessions(provider, evaluators, session_filter=None, *, max_workers=1): composes list_sessions → one Case per session (keyed on session_id) → Experiment, using provider.as_task() as the per-case task. A free function, not a method on Experiment/TraceProvider, to avoid the coupling/circular-dependency issues the issue calls out (alternatives Add styling fixes and license docs. #3 and Add support for dataset generation with strands agents #4).
• Exported SessionFilter and evaluate_sessions from the package root and providers package.

Deviation from the issue sketch

The issue sketched evaluate_sessions(...) -> list[EvaluationReport] with a locally-defined Experiment + task. The current public API is Experiment.run_evaluations, which returns a single EvaluationReport flattened across evaluators (each row tagged via cases[i]["evaluator"]). To match today's API I return one EvaluationReport. Happy to change the shape if you'd prefer otherwise. max_workers defaults to 1 (sequential) because a TracedHandler shares one exporter and some providers share a single network client.

Related Issues

#143

Testing

• hatch test tests/strands_evals/providers/ tests/strands_evals/test_batch.py → 107 passed (includes the existing provider suites, plus new SessionFilter / list_sessions / evaluate_sessions tests).
• hatch test tests/strands_evals/providers/test_trace_provider.py tests/strands_evals/test_batch.py → 21 passed.
• hatch fmt --formatter clean; hatch fmt --linter → all checks passed.
• hatch run hatch-static-analysis:mypy -p src → Success, no issues in 236 source files.

New tests: tests/strands_evals/test_batch.py (batch fan-out, per-session output routing, filter forwarding, empty discovery, NotImplementedError path) and added TestSessionFilter / TestListSessions to tests/strands_evals/providers/test_trace_provider.py.

Checklist

• I have read the CONTRIBUTING doc
• I have added tests that prove my fix/feature works
• New and existing unit tests pass locally
• I have reviewed and understand every line of code in this PR

I use a coding assistant to help implement, and I review and take responsibility for the final change.

[lifelmy]

Task A base refresh: merged current origin/main (24acea695, CI command pinning) without conflicts and pushed fe9fcd6d3 (0 behind / 4 ahead). The provider/session-batch diff is unchanged. Verification: trace-provider/batch suites 21 passed; focused Ruff check/format and git diff --check passed.

[lifelmy]

Task A base refresh: merged current origin/main (9ea55c96d461, README-only) without conflicts and pushed ff575e3b90c8 (0 behind / 5 ahead). The provider/session-batch diff is unchanged. Verification: trace-provider/batch suites 21 passed; repository-wide Ruff check/format and git diff --check passed.

[lifelmy]

Task A base refresh: merged the latest main (9de7900ab) into this branch without conflicts and pushed c170b0ec9 (0 behind).

Verification on the refreshed head:

• uv run pytest -q tests/strands_evals/providers/test_trace_provider.py tests/strands_evals/test_batch.py → 21 passed
• uv run ruff check . → passed
• uv run ruff format --check . → 364 files already formatted
• git diff --check refs/remotes/origin/main...HEAD → passed

I use a coding assistant to help implement, and I review and take responsibility for the final change.