Add upstream provider selection via colon notation (e.g., glm-5:cerebras)

[sh1ftred]

Feature Request: Allow clients to choose upstream provider using colon notation

Problem Statement

Currently, when a client requests a model (e.g., glm-5), the proxy selects the upstream provider(s) automatically based on availability and cost-based routing. However, some clients may prefer to use a specific upstream provider (e.g., Cerebras) when multiple providers support the same model.

Proposed Solution

Introduce a new colon notation that allows clients to explicitly specify their preferred upstream provider:

glm-5:cerebras

This notation tells the proxy:

• glm-5 = the model to use
• cerebras = the preferred upstream provider

The proxy should then:

1. Parse the model string to extract the actual model ID and provider preference
2. If the specified provider supports the model, route to that provider
3. If the specified provider doesn't support the model, return an appropriate error
4. Fall back to automatic selection if no colon notation is provided

Implementation Details

1. Model Parsing: Add a function to parse model strings with optional provider suffix

   • Format: {model_id}:{provider_name}
   • Example: glm-5:cerebras → model_id=glm-5, provider=cerebras

2. Provider Lookup:

   • When a provider is specified, verify it exists and supports the requested model
   • Return error if provider doesn't exist or doesn't support the model

3. Backward Compatibility:

   • Requests without colon notation should work exactly as before
   • Automatic provider selection remains the default behavior

Important: Models with colons in names

Breaking Change Warning

If node runners have models that already contain a colon in their name (e.g., custom/model:v2), this new notation will conflict with the existing model names.

Proposed solution: Add a warning/error when node runners configure models with colons in their names:

Warning: Model names containing colons (e.g., {model_name}) may conflict with the new upstream provider selection feature. Please rename models to avoid colons or use an alias without colons.

This warning should be shown:

• During model configuration/import
• In the admin dashboard
• In the logs when models with colons are loaded

Example Usage

Client request (curl):

curl https://api.routstr.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-cashu: cashuBo2FteCJodHRwczovL21..." \
  -d '{
    "model": "glm-5:cerebras",
    "messages": [{"role": "user", "content": "hello"}]
  }'

Response (success):
Request routed to Cerebras provider.

Response (provider doesn't support model):

{
  "error": {
    "type": "invalid_request_error",
    "code": "provider_not_found_for_model",
    "message": "Provider 'cerebras' does not support model 'glm-5'. Available providers: openai, groq"
  }
}

Labels

• feature
• enhancement

Priority

Medium