[Feature]: optional API Route BYOK provider #452

Activity

1. github-actions commented on Sep 30, 2026

   

   github-actionsbot

   on Sep 30, 2026 – with GitHub Actions

   Contributor

   More actions

   Thanks for the detailed, well-scoped proposal — and for offering to do the implementation work with tests and a changeset.

   This pattern already has precedent in the repo. An opt-in, OpenAI-compatible gateway is not novel here:

   • .changeset/atlascloud-provider.md:8 — "Add Atlas Cloud as a built-in OpenAI-compatible provider for onboarding, validation, model discovery, and provider selection." (this is the Add Atlas Cloud as a built-in provider #404 work you referenced)
   • .changeset/requesty-proxy-preset.md:5 — "Add Requesty as an OpenAI-compatible proxy preset in provider settings."

   The packages/providers/ package is described in the repo docs as the "pi-ai adapter + custom provider extensions" layer, and AGENTS.md states the rule directly: "Register custom providers through pi.registerProvider(). Do not build a parallel provider SDK layer." So an API Route preset that goes through the existing adapter, with no new SDK, no bundled runtime, no app-operated proxy, and locally stored BYOK credentials, is consistent with what is already shipped rather than a new category of feature.

   On your scope questions:

   • Your constraints check lines up with the project's hard constraints (BYOK-only, credentials in local config, no bundled runtimes, no telemetry) — those are the same constraints the Atlas Cloud and Requesty presets operate under.
   • A focused PR (provider preset + configuration tests + a changeset, following the existing .changeset/*-provider.md format) is the right shape. Whether maintainers want API Route specifically built in versus documented as a custom-provider recipe is a maintainer scope call that I can't confirm here — it's best to keep the PR small and opt-in so it can be accepted or declined cleanly.
   • Agreed on not inferring pricing or capabilities for deepseek-v4-flash or any other model. Catalog entries should come from what API Route actually exposes, not be guessed.

   Relevant code / files:

   • packages/providers/ — pi-ai adapter and custom provider extensions (where a preset would live)
   • .changeset/atlascloud-provider.md, .changeset/requesty-proxy-preset.md — the existing preset precedent and changeset shape

   Need more info: (so a review can be accurate rather than assumed)

   • Which wire type should the preset declare — OpenAI-compatible chat completions, or the Responses API?
   • Auth format: is it a Bearer token in the Authorization header, and does the key go anywhere else (e.g. custom headers)?
   • Does API Route expose an OpenAI-compatible /v1/models listing for discovery/validation, or should the catalog be a static model-ID list?
   • Any known streaming/SSE or base-URL path quirks with https://global.api-route.com/v1 that a Test Connection flow should surface?

   Once those are pinned down, a small opt-in preset PR should be straightforward to evaluate against the existing provider registration path.

   ---

   Open-CoDesign Bot
2. DennyHo0917 commented on Sep 30, 2026

   

   DennyHo0917

   on Sep 30, 2026

   Author

   More actions

   Thanks for the pointers. To answer the protocol questions:

   1. Wire type: the proposed preset would use OpenAI-compatible Chat Completions, through the existing pi provider registration path. Configure the SDK base URL as https://global.api-route.com/v1; the request endpoint is POST /v1/chat/completions. This preset would not require Responses API support.

   2. Authentication: Authorization: Bearer <API_ROUTE_API_KEY>. No additional authentication headers, attribution headers, or key in the URL/body are required for this OpenAI-compatible path.

   3. Discovery: authenticated GET https://global.api-route.com/v1/models is supported, with the OpenAI-style object: "list" and data[].id shape. The returned IDs are filtered by the key's group/model permissions and current availability, so discovery is preferable to a frozen catalog. Send the exact returned model ID without adding a provider prefix. A model appearing in the list does not establish tool, vision, context-window, or pricing capabilities; those should not be inferred. I rechecked the public endpoint without a key: it returns HTTP 401 as expected.

   4. Streaming / URL handling: Chat Completions supports stream: true with text/event-stream, JSON data: chunks, and a terminal data: [DONE]. Use /v1 once in the base URL; do not configure the full /chat/completions URL as the SDK base URL. I am not aware of a provider-specific SSE extension needed for this preset. This clarification is based on the service implementation; I have not run an authenticated Open CoDesign streaming test yet. The connection flow should distinguish model discovery from an actual inference check and surface key permissions, quota/balance, and model-availability errors instead of treating discovery alone as proof that generation works.

   Public service overview: https://www.api-route.com/docs/overview. Keys: https://www.api-route.com/tokens. Current catalog: https://www.api-route.com/pricing.

   Could a maintainer confirm whether an optional built-in API Route preset is welcome, or whether you prefer a custom-provider setup guide? Once that scope is confirmed, I can prepare the small preset/configuration-tests/changeset PR following #404.

   Prepared with Codex assistance on behalf of API Route's maintainer.
3. github-actions commented on Sep 30, 2026

   

   github-actionsbot

   on Sep 30, 2026 – with GitHub Actions

   Contributor

   More actions

   Thanks — this answers all four points from the earlier round, and it is the level of detail that makes a provider preset reviewable.

   What matches existing structure (verified):

   • packages/providers/ is described in the repository layout as the "pi-ai adapter + custom provider extensions" layer (CLAUDE.md).
   • AGENTS.md states the rule directly: "Register custom providers through pi.registerProvider(). Do not build a parallel provider SDK layer," and "All LLM calls go through pi-ai; do not import provider SDKs directly in app code." A preset that rides the existing OpenAI-compatible path, with no new SDK, no bundled runtime, and no app-operated proxy, fits that rule.
   • Precedent exists in the changesets: .changeset/atlascloud-provider.md:3 bumps @open-codesign/providers (patch) and :8 describes "a built-in OpenAI-compatible provider for onboarding, validation, model discovery, and provider selection"; .changeset/requesty-proxy-preset.md:5 adds Requesty "as an OpenAI-compatible proxy preset in provider settings."

   On your protocol answers:

   • Chat Completions through the existing pi registration path, Authorization: Bearer <API_ROUTE_API_KEY>, and https://global.api-route.com/v1 as the base URL (request endpoint POST /v1/chat/completions) is the same shape the existing OpenAI-compatible presets use. Nothing there requires Responses API support.
   • Preferring authenticated /v1/models discovery over a frozen catalog lines up with how the Atlas Cloud changeset separates "validation" from "model discovery". Discovery should be treated as a key-scoped list, not a capability source — no tool/vision/context-window/pricing inference.
   • The streaming details (stream: true, text/event-stream, JSON data: chunks, terminal data: [DONE], and /v1 used once in the base URL) are consistent with a standard OpenAI-compatible stream. Since you have not yet run an authenticated Open CoDesign streaming test, a short manual verification note in the PR would help a reviewer.

   On scope (built-in preset vs. custom-provider guide): that is a maintainer decision and I cannot confirm it here. What I can say is that the shape you are proposing — small, opt-in, reusing the existing registration path, with configuration tests and a changeset following the .changeset/*-provider.md format — is the easiest kind to accept or decline cleanly.

   Your point that discovery is not proof of generation is a good one and worth encoding as an acceptance criterion. A successful /v1/models call does not establish quota/balance, key group permissions, or usable capabilities at inference time. A preset PR that keeps model discovery and an actual inference check as distinct steps, and surfaces permissions / quota / model-availability errors rather than treating the list call as success, would be more useful than one that only tests discovery.

   Next step: open the focused PR. Keep it opt-in, follow the existing changeset format, and note your manual verification status so the maintainers can weigh built-in preset versus documented recipe.

   ---

   Open-CoDesign Bot