Skip to content

feat: add http_headers support to LlmProvider for custom upstream headers#870

Open
octo-patch wants to merge 1 commit intokatanemo:mainfrom
octo-patch:fix/issue-592-llmprovider-http-headers
Open

feat: add http_headers support to LlmProvider for custom upstream headers#870
octo-patch wants to merge 1 commit intokatanemo:mainfrom
octo-patch:fix/issue-592-llmprovider-http-headers

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #592

Problem

LlmProvider supports custom endpoint and access_key but has no way to pass custom HTTP request headers to the upstream provider. Some providers or API gateways require non-standard headers for authentication (e.g., X-API-Key, X-Org-ID) or telemetry purposes that cannot be expressed through the existing access_key field.

Solution

Added an optional http_headers field to LlmProvider that accepts a key-value map of HTTP headers to be injected into every upstream request for that provider:

model_providers:
  - model: provider/model-name
    access_key: $API_KEY
    http_headers:
      X-Custom-Header: "custom-value"
      X-API-Version: "v1"

Changes:

  • crates/common/src/configuration.rs: Added http_headers: Option<HashMap<String, String>> field to LlmProvider struct
  • crates/common/src/llm_providers.rs: Updated create_test_provider test helper to include the new field
  • crates/llm_gateway/src/stream_context.rs: Added set_custom_provider_headers() method that applies configured headers after auth headers are set; called from on_http_request_headers
  • config/plano_config_schema.yaml: Added http_headers to the schema for both model_providers and the deprecated llm_providers sections

Custom headers are applied after provider-specific auth headers (Authorization, x-api-key), so they can supplement but not accidentally override authentication.

Testing

The change is backward compatible — the new field is Option<...> and defaults to None, so existing configs are unaffected. The schema addition ensures validation passes for configs using the new field.

…ders (fixes katanemo#592)

Allows users to specify custom HTTP request headers per LLM provider in
plano_config.yaml, useful for providers requiring non-standard auth or
telemetry headers:

  model_providers:
    - model: provider/model
      access_key: $API_KEY
      http_headers:
        X-Custom-Header: "custom-value"
        X-API-Version: "v1"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for custom HTTP request headers in LlmProvider

1 participant