Skip to content

[Feature]: 对齐“测试连接”和“实际调用”链路 / Ensure connection-test and actual-invocation parity #216

Open
#421
Open
[Feature]: 对齐“测试连接”和“实际调用”链路 / Ensure connection-test and actual-invocation parity#216
#421
Labels
enhancementNew feature or requesttriageAwaiting maintainer review

Description

@Sun-sunshine06
Sun-sunshine06
opened on Apr 23, 2026

[Feature]: 对齐“测试连接”和“实际调用”链路 / Ensure connection-test and actual-invocation parity

Problem / pain point / 问题

当前“测试连接”与“实际调用”并不总是走同一套有效 provider contract，因此容易出现误判：

  • 测试连接通过，但真实 generate 失败
  • 测试连接失败，但真实调用其实可用
  • /models 探测、wire 选择、baseUrl 规范化、keyless 判定、OAuth / Bearer 认证处理不完全一致
  • active provider 的解析、renderer 传入值覆盖、真实调用时的 baseUrl snap-to-config 逻辑，并没有完整复用到测试链路

Today the connection-test path and the real invocation path do not always share the same effective provider contract, which creates false positives and false negatives:

  • connection test passes while real generate fails
  • connection test fails even though real invocation works
  • /models probing, wire selection, baseUrl normalization, keyless handling, and OAuth / Bearer auth are not fully aligned
  • active-provider resolution and runtime-only config snapping are not consistently reused by connection tests

Proposed solution / 方案

把“测试连接”升级成“对真实调用链路的低成本验证”，而不是一条独立的近似探测路径。

Turn connection testing into a low-cost validation of the real invocation contract rather than a separate approximate probe.

具体包括：

  • 提炼共享的 “effective provider contract” 解析层
  • 让测试链路与运行链路复用相同的 provider 解析、auth/header、baseUrl、wire、keyless、OAuth 逻辑
  • 对 /models fallback、chat/completions / responses 探测、以及 runtime 实际发送条件做显式对齐
  • 在无法完全复用真实请求时，返回结构化“哪里与真实调用可能分叉”的原因
  • 为“test-pass / runtime-fail”和“test-fail / runtime-pass”建立回归测试

Alternatives considered / 备选方案

  • 继续增强现有 diagnostics，但保留独立探测路径

    • 缺点：可以减少提示不清，但不能消除链路分叉本身

  • 直接用一次完整生成来代替测试连接

    • 缺点：成本高、速度慢、还可能产生副作用

  • Keep improving diagnostics while preserving a separate probe path

    • Rejected because it improves hints but does not remove the divergence

  • Replace connection testing with a full real generate call

    • Rejected because it is slower, costlier, and may have side effects

Scope / 范围

Provider / model

Acceptance criteria / 验收标准

  • connection:v1:test-active 与实际 generate 共享同一套 active provider 解析语义
  • connection:v1:test-provider 与真实运行共享同一套 auth、header、baseUrl、wire、keyless 规则
  • /models fallback 与真实 wire 行为的关系被显式定义，而不是只靠局部 probe 经验
  • ChatGPT Codex OAuth、keyless proxy、OpenAI-compatible gateway 都有 parity 覆盖
  • 至少覆盖以下回归场景：
    • test passes but runtime fails because effective wire / auth / baseUrl differs
    • test fails but runtime succeeds because /models is missing while inference works
  • renderer 能收到结构化结果，明确区分“可连通但与实际调用不一致”

Likely touchpoints / 可能涉及

  • apps/desktop/src/main/connection-ipc.ts
  • apps/desktop/src/main/index.ts
  • apps/desktop/src/main/provider-settings.ts
  • apps/desktop/src/main/resolve-api-key.ts
  • packages/core/src/index.ts
  • packages/providers/src/index.ts

Related issues / 相关 issue

Activity

  1. Sun-sunshine06
    added
    enhancementNew feature or request
    triageAwaiting maintainer review
    on Apr 23, 2026

  2. github-actions commented on Apr 23, 2026

    @github-actions
    github-actionsBot
    on Apr 23, 2026 – with GitHub Actions
    Contributor

    感谢反馈，我核对了当前实现，这个问题描述和代码现状基本一致。当前真实生成链路会先从磁盘里的 canonical active provider 解析出实际使用的 provider / model / baseUrl / wire / httpHeaders，再把这些值传给实际 provider 调用；但连接测试目前仍走独立的 credentials + probe 路径，核心是 GET /models，在部分 wire 上再降级探测 /chat/completions 或 /responses。所以“测试连接”和“实际调用”现在还没有完全共享同一份 effective provider contract。

    Thanks for reporting this. I verified the current code, and the parity gap you described is real today: runtime generation snaps to the canonical active provider config before invoking the provider, while connection testing still uses a separate probe-oriented path.

    项目当前公开版本仍处于 v0.1.x 快速迭代阶段，provider / API 配置链路还在持续收敛（README.md:40, README.md:250）。

    Relevant code:

    • apps/desktop/src/main/provider-settings.ts:215 — resolveActiveModel() is the runtime resolver for the canonical active provider, including baseUrl, wire, httpHeaders, and keyless allowance.
    • apps/desktop/src/main/index.ts:733 — generation snaps to the canonical active provider and ignores stale renderer baseUrl when needed.
    • apps/desktop/src/main/index.ts:815 — runtime passes the resolved baseUrl, wire, and httpHeaders into the actual generate call.
    • apps/desktop/src/main/connection-ipc.ts:699 — connection:v1:test-active uses resolveActiveCredentials() + runProviderTest() instead of the runtime resolver.
    • apps/desktop/src/main/connection-ipc.ts:459 — runProviderTest() is probe-based: GET /models, with degraded fallbacks for openai-chat and openai-responses.
    • apps/desktop/src/main/connection-ipc.ts:428 — ChatGPT Codex connection test is a token-store check, not the same request path as runtime invocation.
    • packages/providers/src/index.ts:286 — runtime-only wire normalization currently strips Gemini models/ prefixes during actual invocation.
    • packages/providers/src/index.ts:341 — runtime-only request shaping also adjusts strict openai-responses payloads and some Claude-compatible identity headers.
    • apps/desktop/src/main/provider-settings.test.ts:323 — there is test coverage for snapping stale hints back to the canonical active provider.
    • apps/desktop/src/main/connection-ipc.test.ts:889 — there is separate test coverage for the degraded /models fallback behavior.

    open-codesign Bot

  3. Sun-sunshine06 commented on Apr 23, 2026

    @Sun-sunshine06
    Sun-sunshine06
    on Apr 23, 2026
    CollaboratorAuthor

    Opened PR #217 for the first parity slice / 已提交 #216 的第一版对齐 PR：#217

  4. Sun-sunshine06 commented on Apr 23, 2026

    @Sun-sunshine06
    Sun-sunshine06
    on Apr 23, 2026
    CollaboratorAuthor

    Correction / 更正：the PR for this work is #220, not #217.\n\nPR link: #220

  5. Sun-sunshine06
    added 2 commits that reference this issue on Apr 24, 2026

    feat(shared,desktop,providers,core): align connection test and invoca…

    ad1136c

    feat(desktop): align connection test credential resolution with runti…

    3bf2add

  6. Sun-sunshine06 commented on Apr 24, 2026

    @Sun-sunshine06
    Sun-sunshine06
    on Apr 24, 2026
    CollaboratorAuthor

    ??????????PR #223 ??? #216 ????????????????????????????????????????????

    ?? PR ?????????

    • ?? provider ????????????? capability ????? Settings ???????????
    • ?? resolved capabilities ? raw explicitCapabilities??? imported official OpenAI/OpenRouter provider ? official base URL ?? reasoning heuristic ???
    • ???????????? runtime / legacy generate / title generation / provider boundary??????? call site ???
    • ? models ???manual/static-hint ????? agent/non-agent ?????? parity ???
    • ??????????? CI ??????

    ?? bot ??????????????????? /models ???? capability profile ?????? wire endpoint ?????Settings ????? ?Connection OK???????????????????????????????? PR ????????????????? PR ????? invocation parity ????

    ??????????
    PR #223 ????? #216 ??????????????????? Settings ? connection parity ?????????????? follow-up?

    PR: #223

  7. Sun-sunshine06
    added a commit that references this issue on Apr 24, 2026

    feat(shared,desktop,providers,core): align connection test and invoca… (

    b93e0dc
  8. hqhq1025
    added
    bot-rerunTemporary label to rerun automation
    and removed
    bot-rerunTemporary label to rerun automation
    on May 1, 2026

  9. github-actions commented on May 1, 2026

    @github-actions
    github-actionsBot
    on May 1, 2026 – with GitHub Actions
    Contributor

    感谢您的更新和 PR #223。

    经核查，PR #223 着重于 provider capability 的解析对齐，它让 Settings 中保存的 capabilities 与 explicitCapabilities 在 provider 边界处共享同一份 resolution 逻辑，这是在走向“测试连接与真实调用共享 effective provider contract”的重要一步。

    不过从 issue #216 的验收标准来看，还有很多未覆盖的部分：

    • baseUrl 的 snap-to-config 逻辑（当 renderer 传入的 baseUrl 与 canonical active provider 不同时，runtime 会忽略 renderer 值，但测试连接并未复用这个规则）
    • auth / header 的完全一致（OAuth / Bearer / keyless）
    • wire 选择、/models fallback 行为与真实调用 wire 的关系
    • codex / openai-compatible relay 的 parity 覆盖
    • test-pass / runtime-fail 和 test-fail / runtime-pass 的回归测试

    目前 PR #223 改进了 packages/providers/src/index.ts 中的 capability 合并，但 issue 描述的完整链路对齐（apps/desktop/src/main/connection-ipc.ts 与 apps/desktop/src/main/index.ts 中的 runtime 路径）仍需后续 PR 解决。

    建议将此 issue 保持开放，作为 tracking epic 直到所有验收条件达成。后续可以在 issue 中关联具体 PR 的剩余部分。

    Open-CoDesign Bot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesttriageAwaiting maintainer review

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions