Add BackgroundTools capability

[DouweM]

Closes #221.

Summary

Adds the BackgroundTools capability that runs selected tools as fire-and-forget asyncio tasks. The agent receives an immediate acknowledgment string and keeps working; when the task completes, its result is delivered as a follow-up message via Pydantic AI's pending message queue (in-flight in pydantic/pydantic-ai#4980).

from pydantic_ai import Agent
from pydantic_ai_harness import BackgroundTools

agent = Agent('openai:gpt-5', capabilities=[BackgroundTools()])

@agent.tool_plain(metadata={'background': True})
async def slow_research(query: str) -> str:
    return await do_expensive_research(query)

Selector

Mirrors CodeMode's tools= API — accepts the standard ToolSelector:

• dict (default {'background': True}) — by metadata
• Sequence[str] — by name
• Callable[(ctx, td), bool] — predicate
• 'all' — every tool

Composes with SetToolMetadata and FunctionToolset.with_metadata(...) to mark a whole MCP server / toolset as background.

Implementation notes

• Per-run state via for_run so concurrent runs don't share tasks
• wrap_tool_execute spawns the task and returns the ack
• after_node_run waits on at least one task before letting End propagate, so the core PendingMessageDrainCapability (which runs after us in reverse order via wrapped_by=[PendingMessageDrainCapability]) finds a follow-up to drain
• wrap_run finally cancels remaining tasks; asyncio.CancelledError does not produce a spurious failure follow-up

Dependency note

[tool.uv.sources] points pydantic-ai-slim at the background-tools branch (where the pending message queue lives) until pydantic/pydantic-ai#4980 merges. We'll switch back to main once that lands.

Test plan

• 6 tests covering: default metadata selector path (success / failure / unmarked / cancellation), name list selector, custom metadata key selector
• 100% line + branch coverage on the new files
• make lint && make typecheck && make test clean

[devin-ai-integration]

Devin Review found 3 potential issues.

View 4 additional findings in Devin Review.

[devin-ai-integration]

🟡 handler: Any in wrap_run violates "no Any types" rule

The AGENTS.md coding standard explicitly requires "pyright strict mode -- no Any types, full type annotations". In wrap_run, the handler parameter and return type both use Any (lines 157-158), whereas the analogous wrap_tool_execute method correctly imports and uses the specific WrapToolExecuteHandler type from pydantic_ai.capabilities.abstract (_capability.py:21). There should be a corresponding WrapRunHandler type (or equivalent) imported and used here instead of Any.

Prompt for agents

In pydantic_ai_harness/background_tools/_capability.py, the wrap_run method uses `handler: Any` and `-> Any` on lines 157-158. The AGENTS.md rule mandates no Any types. The wrap_tool_execute method already correctly imports WrapToolExecuteHandler from pydantic_ai.capabilities.abstract. Check pydantic_ai.capabilities.abstract for a WrapRunHandler type (or similar) and import it in the TYPE_CHECKING block at line 21 to replace the Any usages. The return type should also be specified concretely (likely the agent run result type from the abstract base class signature).

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Tests use FunctionModel instead of TestModel violating testing pattern rule

The AGENTS.md testing patterns section explicitly states: "Use pydantic_ai.models.TestModel for all tests (no real API calls)". All 6 test methods in test_background_tools.py use FunctionModel exclusively, while the existing code_mode tests primarily use TestModel and only use FunctionModel for 2 specific integration tests. While FunctionModel doesn't make real API calls, it directly violates the stated rule. TestModel should be the primary model used, with FunctionModel reserved for tests that absolutely need hand-driven model responses.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🚩 pydantic-ai-slim pinned to feature branch background-tools instead of main

The pyproject.toml source for pydantic-ai-slim was changed from branch = 'main' to branch = 'background-tools'. This is clearly intentional for development of this feature (the capability depends on APIs in that branch), but this PR should not be merged until the background-tools branch is merged into main in the pydantic-ai repo, otherwise the published package would depend on a feature branch that could be deleted or force-pushed.

---

Was this helpful? React with 👍 or 👎 to provide feedback.