Fix annotations list hiding every pre-project-scoping review

[nicolai-rhesis]

Purpose

The Annotations page shows 0–0 of 0 while the Test Runs grid shows review counts on nearly every run, and the Architect reports "no annotations" on runs that visibly have them — then invents reviewer names, comment text and turn numbers when it does have annotation data. Two independent bugs, one per symptom.

Nothing is listed. list_annotations scoped test results with strict project_id = :project_id. Migration a1b2c3d4e5f0 added project_id as a nullable column with no backfill, so every test result predating project scoping carries a NULL project_id permanently — and nothing has assigned one since. Everywhere else in the app those rows are visible: the ORM auto-filter (models/scope_events.py) and the project_isolation RLS policy both apply project_id = :pid OR project_id IS NULL, which is why the test run page counts their reviews via reviewed_tests. Only this query disagreed, so the annotations list came back empty for the entire historical dataset. This is not an edge case — for an org whose reviewed runs all predate the migration, the feature has never returned anything. trace.project_id is NOT NULL, so the trace branch needs no equivalent allowance.

The content is invented. Every list-shaped tool result passes through _compact_list_result_for_history before reaching the prompt. That renderer builds each line from name/title, id and description. An annotation has none of them — the human's words live in comments, the key is review_id, and the inherited id is always null. A real production payload:

{"id": null, "nano_id": null, "review_id": "e87b6bc0-…", "comments": "Test review",
 "status": {"name": "Pass"}, "user": {"name": "Nicolai Bohn"}, "behavior_name": "Off-Domain Request Redirect"}

rendered to the LLM as List response: 1 item(s) followed by a single line reading - ?. Told that reviews exist but shown none of their content, the model fills the gap from imagination. This is data starvation, not creativity: restore the payload and there is nothing left to guess at.

Both were confirmed against production. Worker logs for the reported session show list_annotations called five times, all succeeding, none erroring — ruling out a 400 (missing project scope), a 422 (URL passed instead of a UUID) and any permission problem. The model then fell back to list_test_results → get_test_result, which returns test_reviews inline; that is where the real review text it quoted came from. A freshly created review on a project-stamped test result comes back from the endpoint correctly, isolating NULL project_id as the discriminator.

What Changed

• services/annotations.py: the test-result branch now admits tr.project_id IS NULL, matching the ORM auto-filter and the RLS policy. Org isolation is untouched, and the trace branch is deliberately left strict.
• architect/agent.py: items without a name/title render as their own trimmed JSON — empty values dropped, long strings clipped, non-ASCII left readable — instead of collapsing to - ?. Named entities keep the existing compact one-line form, so list_metrics and friends are unaffected. This also fixes list_test_results, which has no name either and was collapsing to - ? (id: …).
• architect/agent.py: tool results cut at the 4000-char preview now say so. An unmarked cut reads as a complete record, and a large get_test_result easily exceeds it.

Additional Context

• The test run Reviews tab never used this endpoint — it counts reviews straight off test_result.test_reviews (test-run-summary-utils.ts), and the runs grid uses counts.reviewed_tests from result_processor.py. Both go through the ORM, which is why the UI and the API disagreed with no error anywhere.
• Every existing test in tests/backend/routes/test_annotations.py created test results with an explicit project_id, which is exactly why the NULL case slipped through.
• Not addressed here: whether historical test results should be backfilled with a project (derivable via test_run → test_configuration → endpoint.project_id). The OR IS NULL predicate is correct on its own merits — it is what the ORM and RLS already do — but a backfill would additionally stop those rows appearing in every project's view. That is a separate change with its own migration and risk.
• BaseAgent._format_history in the SDK has the same unmarked 4000-char truncation. Left alone to keep this focused on the architect path; worth a follow-up for the agents using the base implementation.
• No migration, no API-contract change. The annotations list can now return rows it previously hid — that is the fix, not a regression.

Testing

cd apps/backend && uv run pytest ../../tests/backend/routes/test_annotations.py   # 19 passed
cd sdk && uv run pytest ../tests/sdk/agents/test_architect.py                      # 157 passed

test_pre_project_scoping_rows_are_visible seeds a run with no ambient project scope, so auto-stamp leaves project_id NULL exactly as pre-migration rows are, then asserts the review comes back under an active project. Reverting the one-line SQL change makes it fail with assert set() == {'e8c33efb-…'} — the endpoint returning nothing, reproducing the production symptom exactly.

TestCompactUnnamedListResults asserts that annotation comments, reviewer, verdict and behavior all survive the renderer and that no line collapses to - ?, with an end-to-end case through _format_history(). TestToolResultTruncationMarker covers the truncation notice.

To verify by hand: open the Annotations page on an org whose reviewed runs predate project scoping. It should list reviews the Test Runs grid already counts. Then ask the Architect about one of those runs and confirm it both finds the reviews and quotes them accurately.

[peqy]

Looks good. The SQL scoping change aligns /annotations with the ORM/RLS NULL-project behavior, and the Architect history rendering now preserves unnamed records (plus marks truncation) to avoid data-starvation hallucinations. Ship it.

[peqy]

Looks good. The SQL scoping change matches the ORM/RLS behavior for org-level (NULL project_id) test results, and the Architect history renderer now preserves unnamed records and marks truncation to avoid data-starvation hallucinations. Tests cover both regressions. Ship it.