Skip to content

feat(session): retry transient provider failures - #70

Merged
pepicrft merged 2 commits into
mainfrom
feat/retry-policy
Jun 12, 2026
Merged

feat(session): retry transient provider failures#70
pepicrft merged 2 commits into
mainfrom
feat/retry-policy

Conversation

@pepicrft

Copy link
Copy Markdown
Collaborator

Summary

  • Wraps each LLM call in the native runtime with a bounded retry policy (default: 3 attempts, exponential backoff + jitter on HTTP 408/425/429/500/502/503/504 and stream errors wrapping those).
  • Streaming calls only retry while no event has been emitted to the caller, so partial output is never silently duplicated. Once a chunk has shipped, the underlying error is surfaced and the caller decides how to recover.
  • Configurable via the :retry option on Condukt.run/3, Condukt.stream/3, and start_link/1 (boolean, keyword list, or %Condukt.Retry{} struct). Also accepts a custom :classify callback.

Why

In production we just hit a 503 service overloaded from an upstream LLM in the middle of a streaming Slack reply. The session bubbled the error straight to the user as a generic fallback message even though the call was a textbook transient failure. Retrying once at the provider boundary would have masked it entirely; in the steady state every Condukt consumer wants the same behavior.

Test plan

  • mix test — 405 passed, 37 excluded
  • mix credo lib/condukt/retry.ex lib/condukt/session.ex lib/condukt.ex — only the pre-existing 32-field struct warning
  • mix format --check-formatted clean for changed files
  • Wire it through Atlas (separate PR) and confirm an injected 503 retries and recovers

🤖 Generated with Claude Code

Wrap each LLM call in the native runtime with a bounded retry policy
(default: 3 attempts, exponential backoff + jitter). The policy
classifies HTTP 408, 425, 429, 500, 502, 503, 504, and stream errors
that wrap one of those statuses, as retryable.

Streaming calls only retry while no event has been emitted to the
caller. Once a :text or :thinking chunk has reached the subscriber, the
underlying error is surfaced so the caller can decide how to recover
without silently duplicating partial output.

Configurable via the :retry option on Condukt.run/3, Condukt.stream/3,
and start_link/1 (boolean, keyword list, or %Condukt.Retry{} struct).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pepicrft pepicrft self-assigned this Jun 12, 2026
Folds :runtime_opts back into :runtime as a {module, opts} tuple so the
Session struct stays under credo's 32-field warning threshold after the
:retry field was added in the previous commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pepicrft
pepicrft requested review from a team, esnunes and fortmarek and removed request for a team June 12, 2026 12:43
@pepicrft
pepicrft merged commit e5e9ad2 into main Jun 12, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants