Skip to content

DX-3101: fix Pi, Goose and Aider custom harness examples - #256

Open
alitariksahin wants to merge 2 commits into
mainfrom
DX-3101
Open

alitariksahin wants to merge 2 commits into
mainfrom
DX-3101

Conversation

@alitariksahin

Copy link
Copy Markdown
Collaborator

Fixes found while running every custom harness example against a real box.

Pi (was broken: both turns returned an empty result)

  • getModel moved off the @earendil-works/pi-ai root in 0.80, so the unpinned install crashed on import. Now pins @earendil-works/pi-coding-agent@0.99.2 and resolves models with ModelRuntime
  • MCP never worked: extensionUrls isn't a createAgentSession option. Box's servers are now written to the session's mcp.json (npm and URL, with headers, direct exposure) and loaded by Pi's MCP extension; PI_CODING_AGENT_DIR points the extension at the session dir
  • Images are passed natively (prompt(text, { images })); other binary files are saved under .box-attachments
  • Model errors (bad key, rate limit) end the run with an error event; Pi reports them on the message instead of throwing
  • Tool results were [object Object]; now the text content
  • Dropped the synthetic pi_agent tool event and the stack trace in error messages

Goose

  • No tool events were emitted: Goose 1.52 sends tool calls as toolRequest / toolResponse message parts, not tool_call events
  • Startup banner (__( O)> new session…) no longer lands in the result
  • Reports cache tokens and cost_usd; result is the final answer, with a paragraph break between text before and after tool calls
  • Image and other binary attachments are saved and referenced instead of skipped

Aider

  • Banners, privacy notice, model lines and token lines no longer land in the result (--no-check-update --no-show-release-notes --no-analytics, the rest filtered; COLUMNS widened so long lines don't wrap past the filter)
  • Aider exits 0 on model errors; a litellm.*Error with no successful call now fails the run
  • Token lines are summed per run and cache hits reported as cached tokens
  • Image attachments are passed with --read
  • Claude models newer than aider 0.86.2 (e.g. claude-sonnet-5-5) reject the temperature it sends; for models missing from aider's bundled model-settings.yml the harness writes a settings entry with use_temperature: false

Testing

Live against Box for all three: streaming with tool events, follow-up session, token counts, MCP (npm server-everything + context7 URL; not Aider, which has no MCP), text + two same-named images, pause/resume, configureModel() before a follow-up, invalid key fails the run. All pass, except that Aider emits no tool events (it doesn't call tools). Each example file also ran unchanged end to end.

Not fixed here (box-backend): a harness that exits non-zero without a done/error event is reported as a completed run with an empty result, which is why the broken Pi example looked like it worked.

🤖 Generated with Claude Code

Pi: pin pi-coding-agent 0.99.2 and use ModelRuntime (getModel left the
pi-ai root in 0.80, so the example crashed on import); load MCP servers
through Pi's MCP extension (extensionUrls never existed); pass images
natively; fail the run on model errors; fix tool result output.

Goose: map toolRequest/toolResponse message parts to tool events (Goose
has no top-level tool_call events); report cache tokens and cost; keep
the startup banner out of the result; save image attachments.

Aider: keep banners and token lines out of the result; fail the run on
litellm errors (aider exits 0); sum token lines including cache hits;
pass image attachments with --read; drop temperature for Claude models
this aider release doesn't know.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Oct 1, 2026

Copy link
Copy Markdown

DX-3101

…eparately

Pi called process.exit() right after writing done, which could cut off a
large result on a pipe; emit now resolves once the event is written.
Aider prints the cost on its own line when there are cache reads and
writes; cost lines are now parsed and filtered independently of the
token line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant