Expose external provider models through GET /v1/models - #409
Open
urucoder wants to merge 1 commit into
Open
Conversation
Co-authored-by: urucoder <37780362+urucoder@users.noreply.github.com>
Author
|
Closes #410 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Controllers with configured external OpenAI-compatible providers only surfaced those providers' models via
GET /studio/provider-models. Since models are discovered throughGET /v1/models, external provider models were invisible in the model picker even when the provider was configured and reachable.Changes
discoverProviderModelsandenabledProvidersWithApiKeyintocontroller/src/services/provider-routing.ts, replacing the inlineproviderModelsfunction previously duplicated inprovider-routes.ts./studio/provider-models: now consumes the shared helper; behavior unchanged.GET /v1/models: now also queries enabled, API-key-configured providers and appends their models as qualified IDs (<provider-id>/<upstream-model-id>), alongside existing recipe/active-inference entries. Per-provider discovery failures are swallowed (Effect.option) so one unreachable provider never fails the whole endpoint, matching/studio/provider-modelssemantics.openai-routes.ts) — it already parses the qualified ID and rewrites the upstream request model correctly.Example: with provider
my-provider(base_urlhttp://<host>:8000) enabled and reportingexample-model-1,GET /v1/modelsnow includes:{ "id": "my-provider/example-model-1", "object": "model", "owned_by": "my-provider", "active": false, "metadata": { "vision": false } }Validation
Manually verified end-to-end against a live controller and a mock upstream provider server:
GET /v1/modelsreturned the qualified model ID, and a chat completion request using that ID was routed upstream with the model field rewritten back to the original provider model ID.UI changes
None — controller-only change. Effect is a discoverability fix for the existing model picker.
Risks / rollout notes
/v1/modelsresponse; existing consumers unaffected./v1/modelsrequest (10s timeout, best-effort, run concurrently) — negligible latency impact for typical provider counts.Closes #410