[lifelmy]

Description

Adds ExperimentTrendAnalyzer, a pure analysis layer that connects overall_score and pass rate across sequential experiment runs. Today each run_evaluations() produces a fresh EvaluationReport with no link to prior runs, so a trajectory like 0.91 -> 0.85 -> 0.78 -> 0.71 yields no signal. This analyzer fits an OLS slope (via statistics.linear_regression) to each metric over a window of runs, labels the direction (improving / degrading / stable with a small epsilon so run-to-run jitter is not a false trend), and flags any_regression when either metric is trending down.

Key points:

• New standalone module src/strands_evals/trend.py — ExperimentTrendAnalyzer, ExperimentTrendReport, MetricTrend, RunPoint. No changes to EvaluationReport, Experiment, or EvaluationDataStore, so nothing existing changes behavior.
• Two constructors: from in-memory EvaluationReports (run order, optional run_ids), or from_files(...) reading JSON written by EvaluationReport.to_file (with an optional window to keep the most recent N runs).
• Pass rate is derived from test_passes, falling back to overall_score when a report has none; a single run is reported as stable with a zero slope.
• to_dict() / to_file() for downstream reporting.
• Exported from the package __init__.

Related Issues

Closes #186

Type of Change

New feature

Testing

hatch test tests/strands_evals/test_trend.py -> 28 passed. Also ran hatch test tests/strands_evals/test_trend.py tests/strands_evals/types/ -> 132 passed (no regressions). hatch fmt --formatter --check -> 362 files already formatted; hatch fmt --linter -> all checks passed; mypy -p src via hatch-static-analysis env -> Success, no issues in 236 source files.

• I ran the linter, formatter, type checker, and the test suite for the affected package

Checklist

• I have read the CONTRIBUTING document
• I have reviewed and understand every line of code in this PR. I use a coding assistant to help implement, and I review and take responsibility for the final change.
• My change is focused and reasonably small; I have split unrelated work into separate PRs
• I have added any necessary tests that prove my feature works
• I have updated the documentation accordingly (new public API; happy to add docs if maintainers point me at the docs source)
• My changes generate no new warnings

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[lifelmy]

Task A base refresh: merged current origin/main (24acea695, CI command pinning) without conflicts and pushed a24be05f6 (0 behind / 4 ahead). The trend-analyzer diff is unchanged. Verification: trend suite 28 passed; focused Ruff check/format and git diff --check passed.

[lifelmy]

Task A base refresh: merged current origin/main (9ea55c96d461, README-only) without conflicts and pushed ee0e3c760730 (0 behind / 5 ahead). The trend-analyzer diff is unchanged. Verification: trend suite 28 passed; repository-wide Ruff check/format and git diff --check passed.

[lifelmy]

Task A base refresh: merged the latest main (9de7900ab) into this branch and pushed 4fa01339b (0 behind).

This was a real top-level export conflict in src/strands_evals/__init__.py; I resolved it by preserving this PR's trend-analysis exports alongside upstream's new StructuredOutputReport export.

Verification on the refreshed head:

• uv run pytest -q tests/strands_evals/test_trend.py tests/strands_evals/evaluators/deterministic/test_structured_output.py → 28 passed, 188 skipped
• uv run ruff check . → passed
• uv run ruff format --check . → 364 files already formatted
• git diff --check refs/remotes/origin/main...HEAD → passed

I use a coding assistant to help implement, and I review and take responsibility for the final change.