Skip to content

Conversation

@scrrlt
Copy link
Contributor

@scrrlt scrrlt commented Jan 29, 2026

Key changes
Client & factory:

  • LLMClient.complete now forwards history as messages (prevents context drop).
  • Factory preserves provider context for local routes in error mapping (so metrics/logs show provider).

Streaming & concurrency:

  • Routing stream now enforces circuit breaker + TrafficController (bulkheads + rate token bucket).
  • TrafficController guards against invalid RPM (requests_per_minute <= 0).
  • Cohere streaming is rejected explicitly (LLMConfigError) until proper parser support is added.

Providers & SDKs:

  • Anthropic provider: full integration (complete + stream), sanitized kwargs, stream handling.
  • Local provider: local LLM server integration and streaming support.
  • OpenAI provider: warn and disallow DISABLE_SSL_VERIFY in production (prevents accidental MITM).
  • Base provider: treat LLMAPIError.status_code None as retriable/recordable.

Error mapping, telemetry, and utils:

  • map_error improvements and tests.
    is_local_llm_server handles schemeless URLs and private IP detection.
  • Circuit breaker logging improved and unexpected-state visibility added.
    CI & lint:
  • .github/workflows/tests.yml: fail fast (removed "|| echo ..."), un-ignored tests/agents.
  • pyproject.toml: re-enable B006 and stop ignoring B904 (catch mutable-default bugs and force exception chaining).
  • pre-commit/ruff formatting applied and committed.

Touches files in LLM

ruff: passed (files touched only)
pre-commit: ran and auto-fixed formatting
mypy --strict: no errors for targeted modules
pytest tests/services/llm: 42 passed (core tests)
pytest tests/agents: agent tests pass (now included in CI)
bandit: ran across changed files (low-level findings limited to asserts in tests — expected)

Behavioral notes & compatibility

  • Non‑breaking for most users — behavior improves reliability. Minor, intended behavior changes:
  • Cohere streaming now raises LLMConfigError (explicit rejection) until supported.
  • DISABLE_SSL_VERIFY in production now raises LLMConfigError (prevents unsafe production config).
    Developer-facing rule changes:
  • B006 enabled prevents mutable default hazards.
  • B904 removed from ignore enforces exception chaining where appropriate.

* Core LLM hardening (slim): essential fixes and tests only

* Add Anthropic and Local provider implementations
@pancacake
Copy link
Collaborator

Thanks bro!!!

@pancacake pancacake merged commit 94fe5f2 into HKUDS:dev Jan 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants