Conversation
Implements a capability that dynamically adjusts model thinking effort based on task complexity signals via `get_model_settings()` returning a callable. Built-in heuristic uses high effort on first step and after tool errors, low effort on simple follow-ups. Supports custom effort functions for domain-specific logic. Closes #84 Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
- Step 2 now returns medium (was incorrectly going straight to low) - Step 3+ returns low as intended - Removed unreachable default branch - Added many-tool-calls signal: if the last ModelResponse had 3+ ToolCallParts, use medium effort regardless of step number Co-Authored-By: Claude Opus 4.6 (1M context) <[email protected]>
DouweM
requested review from
Kludex,
adtyavrdhn,
dmontagu,
dsfaccini and
samuelcolvin
as code owners
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| """Built-in heuristic effort selector. | ||
|
|
||
| Rules (evaluated in order): | ||
| 1. First step (``run_step == 1``): ``'high'`` -- understand the task. |
There was a problem hiding this comment.
🟡 Docstring says run_step == 1 but code uses run_step <= 1
The docstring for default_effort_fn documents rule 1 as "First step (run_step == 1)" but the actual implementation at line 68 uses ctx.run_step <= 1, which also matches run_step == 0. The test test_step_zero_high (tests/test_adaptive_reasoning.py:152-154) confirms that run_step=0 returns 'high', matching the code but not the docstring. The PLAN.md:38 correctly documents this as run_step <= 1.
| 1. First step (``run_step == 1``): ``'high'`` -- understand the task. | |
| 1. First step (``run_step <= 1``): ``'high'`` -- understand the task. |
Was this helpful? React with 👍 or 👎 to provide feedback.
| @dataclass | ||
| class AdaptiveReasoning(AbstractCapability[Any]): | ||
| """Dynamically adjusts model thinking effort per step. | ||
|
|
||
| By default a built-in heuristic is used: | ||
|
|
||
| * **First step** -> ``'high'`` (understand the task) | ||
| * **After tool errors** -> ``'high'`` (reason about what went wrong) | ||
| * **Simple follow-ups** -> ``'low'`` (just incorporating tool results) | ||
|
|
||
| Supply a custom ``effort_fn`` to override these rules:: | ||
|
|
||
| def my_effort(ctx: RunContext[MyDeps]) -> Literal['low', 'medium', 'high']: | ||
| if ctx.run_step > 5: | ||
| return 'high' # wrap-up needs careful thought | ||
| return 'medium' | ||
|
|
||
| agent = Agent(..., capabilities=[AdaptiveReasoning(effort_fn=my_effort)]) | ||
| """ | ||
|
|
||
| effort_fn: Callable[[RunContext[Any]], Literal['low', 'medium', 'high']] = field(default=default_effort_fn) | ||
| """Callable that receives the current ``RunContext`` and returns an effort level.""" | ||
|
|
||
| def get_model_settings(self) -> Callable[[RunContext[Any]], ModelSettings]: | ||
| """Return a dynamic model-settings callable that sets ``thinking`` per step.""" | ||
|
|
||
| def _resolve(ctx: RunContext[Any]) -> ModelSettings: | ||
| effort = self.effort_fn(ctx) | ||
| return ModelSettings(thinking=_EFFORT_TO_THINKING[effort]) | ||
|
|
||
| return _resolve |
There was a problem hiding this comment.
🚩 Capability only overrides get_model_settings, no other hooks
The AdaptiveReasoning class only implements get_model_settings() from the AbstractCapability interface. Without being able to inspect the AbstractCapability source (the pydantic-ai-slim package isn't installed in the review environment), I could not verify whether there are other required or recommended methods (e.g., get_system_prompt, get_tools). The tests do verify isinstance(cap, AbstractCapability) passes, and the tests presumably run in CI, so this is likely fine.
Was this helpful? React with 👍 or 👎 to provide feedback.
Audit vs prior art: AdaptiveReasoningWorth adding now:
Follow-up opportunities:
|
Audit vs prior art: AdaptiveReasoningWorth adding now:
Follow-up opportunities:
|
DouweM
removed request for
Kludex,
adtyavrdhn,
dmontagu,
dsfaccini and
samuelcolvin
1 participant
Summary
AdaptiveReasoningcapability that dynamically adjusts model thinking effort per step usingget_model_settings()with a callableeffort_fn(RunContext) -> Literal['low', 'medium', 'high']for domain-specific logicpydantic_harnesspackageTest plan
_has_tool_errorshelper,default_effort_fnheuristic,AdaptiveReasoningcapability with default and custom effort functionsCloses #84
🤖 Generated with Claude Code