Problem
Nothing in model-manager's API says which API interfaces a served model answers. Model, LoadedModel and Preset carry capabilities (the preset's free-form tags: chat, tools, reasoning, …) and format (the string vLLM); the agent endpoint hardcodes Provider: "OpenAI" and BaseURL + "/v1" (internal/backend/kserve/served.go). A caller — the portal's Models pages through muster, an agent picking a model, a person pointing a CLI at it — cannot see that a model served on llm-d answers OpenAI chat completions, legacy completions, the Responses API and Anthropic Messages with count_tokens, or that an embeddings preset answers /v1/embeddings and no chat interface at all. Clients differ exactly here (Claude Code speaks Messages, most CLIs chat completions, newer ones Responses), so the guess is made on the wrong side.
The interface set is not a preset property and must not be typed per preset. It follows the runtime version (vLLM serves /v1/responses since 0.10.0, /v1/messages since 0.11.1, /v1/messages/count_tokens since 0.17.0) and the model's task class: since vLLM 0.16.0 the OpenAI-compatible server registers a family's routers only when the engine's supported tasks include it — a generate model gets chat completions, completions, Responses and Messages, a pooling model gets embeddings, score and rerank and no /v1/chat/completions (a request to it is a plain 404). GET /v1/models (id, root, max_model_len, permission) and GET /version say nothing about tasks; GET /openapi.json (FastAPI's default, absent only with --disable-fastapi-docs) lists exactly the routes this server registered for this model. The llm-d runtime the platform ships is well past 0.16.
Proposed solution
When an LLMInferenceService turns Ready, the kserve backend reads the running server once, at the origin the agent endpoint uses (the workload Service, port 8000): GET /version for the runtime version and GET /openapi.json for the route list. The route list is the interface list. model-manager maps the known paths onto the platform's interface vocabulary — agentgateway's format names, so the same list can later describe the model to a gateway — and reports:
runtime: {name: vllm, version: "0.23.0"}
interfaces:
- {type: Completions, path: /v1/chat/completions} # legacy /v1/completions implied
- {type: Responses, path: /v1/responses}
- {type: Messages, path: /v1/messages}
- {type: AnthropicTokenCount, path: /v1/messages/count_tokens}
# a pooling preset: {type: Embeddings, path: /v1/embeddings}
on LoadedModel (and on Model while it is loaded), in list_loaded_models, get_model, GET /api/v1/loaded and api/openapi.yaml. Preset gets no interface field: a preset states no interfaces. The read happens once per Ready transition and is kept with the loaded model. A server that answers no /openapi.json (a runtime started with --disable-fastapi-docs) or is older than 0.16 reports an empty list and a reason naming why — nothing is inferred from a version table.
Whether a registered interface works for a given request is a launch-time fact of the preset — tool calling needs --enable-auto-tool-choice --tool-call-parser, thinking blocks a --reasoning-parser, chat a template — and stays with the preset (giantswarm/agent-platform#313); this issue reports what the server registered, not what a request will get back.
The read needs the serving namespace's network policy to admit model-manager and the presets to keep FastAPI's docs on: giantswarm/agent-platform#602.
Acceptance criteria
Epic: giantswarm/giantswarm#37590. Related: giantswarm/agent-platform#602 (the policies and the docs flag), giantswarm/backstage#2495 (the Models pages), #145 (the gateway model built from this list), giantswarm/agent-platform#313 (the parsers stay preset facts).
Problem
Nothing in model-manager's API says which API interfaces a served model answers.
Model,LoadedModelandPresetcarrycapabilities(the preset's free-form tags:chat,tools,reasoning, …) andformat(the stringvLLM); the agent endpoint hardcodesProvider: "OpenAI"andBaseURL + "/v1"(internal/backend/kserve/served.go). A caller — the portal's Models pages through muster, an agent picking a model, a person pointing a CLI at it — cannot see that a model served on llm-d answers OpenAI chat completions, legacy completions, the Responses API and Anthropic Messages withcount_tokens, or that an embeddings preset answers/v1/embeddingsand no chat interface at all. Clients differ exactly here (Claude Code speaks Messages, most CLIs chat completions, newer ones Responses), so the guess is made on the wrong side.The interface set is not a preset property and must not be typed per preset. It follows the runtime version (vLLM serves
/v1/responsessince 0.10.0,/v1/messagessince 0.11.1,/v1/messages/count_tokenssince 0.17.0) and the model's task class: since vLLM 0.16.0 the OpenAI-compatible server registers a family's routers only when the engine's supported tasks include it — a generate model gets chat completions, completions, Responses and Messages, a pooling model gets embeddings, score and rerank and no/v1/chat/completions(a request to it is a plain 404).GET /v1/models(id, root, max_model_len, permission) andGET /versionsay nothing about tasks;GET /openapi.json(FastAPI's default, absent only with--disable-fastapi-docs) lists exactly the routes this server registered for this model. The llm-d runtime the platform ships is well past 0.16.Proposed solution
When an
LLMInferenceServiceturns Ready, the kserve backend reads the running server once, at the origin the agent endpoint uses (the workload Service, port 8000):GET /versionfor the runtime version andGET /openapi.jsonfor the route list. The route list is the interface list. model-manager maps the known paths onto the platform's interface vocabulary — agentgateway's format names, so the same list can later describe the model to a gateway — and reports:on
LoadedModel(and onModelwhile it is loaded), inlist_loaded_models,get_model,GET /api/v1/loadedandapi/openapi.yaml.Presetgets no interface field: a preset states no interfaces. The read happens once per Ready transition and is kept with the loaded model. A server that answers no/openapi.json(a runtime started with--disable-fastapi-docs) or is older than 0.16 reports an empty list and areasonnaming why — nothing is inferred from a version table.Whether a registered interface works for a given request is a launch-time fact of the preset — tool calling needs
--enable-auto-tool-choice --tool-call-parser, thinking blocks a--reasoning-parser, chat a template — and stays with the preset (giantswarm/agent-platform#313); this issue reports what the server registered, not what a request will get back.The read needs the serving namespace's network policy to admit model-manager and the presets to keep FastAPI's docs on: giantswarm/agent-platform#602.
Acceptance criteria
list_loaded_modelson a Ready llm-d model carriesruntime.version(vLLM's) andinterfaceswithCompletions,Responses,MessagesandAnthropicTokenCountfor a generate preset; an embeddings preset listsEmbeddingsand noCompletions./openapi.jsonis unreachable or absent reports an empty list and a reason — no per-version table, nothing invented.list_modelsshows the same for a loaded model;list_presetscarries no interface field.openapi.jsondocuments of a generate and a pooling server; a lab proof in the agentlab serving slice with the fields on the wire through muster.Epic: giantswarm/giantswarm#37590. Related: giantswarm/agent-platform#602 (the policies and the docs flag), giantswarm/backstage#2495 (the Models pages), #145 (the gateway model built from this list), giantswarm/agent-platform#313 (the parsers stay preset facts).