Skip to content

[feat][evaluation] support a2a_agent and custom_agent eval target types on the OpenAPI experiment-create path - #560

Merged
jamesonics merged 1 commit into
mainfrom
feat/openapi-eval-target-a2a-custom
Jun 24, 2026
Merged

jamesonics merged 1 commit into
mainfrom
feat/openapi-eval-target-a2a-custom

Conversation

@jamesonics

Copy link
Copy Markdown
Collaborator

What type of PR is this?

feat

Check the PR title

  • This PR title match the format: [][] .
  • The description of this PR title is user-oriented and clear enough for others to understand.
  • Add documentation if the current PR requires user awareness at the usage level.
  • This PR is written in English. PRs not in English will not be reviewed.

(Optional) Translate the PR title into Chinese

[feat][evaluation] OpenAPI 创建实验链路支持 a2a_agent 与 custom_agent 评测对象类型

(Optional) More detailed description for this PR

en:
The web link already supports two eval target types — A2A Agent (a2a_agent, type=9) and the in-house
long-connection agent (custom_agent, type=10) — and their execution operators are registered downstream. The
OpenAPI experiment-create path, however, rejected them: mapOpenAPIEvalTargetType only handled 6 types, so
a2a_agent / custom_agent fell through to default and were blocked both by the up-front
SubmitExperimentOApi validation and by the convertor.

This PR wires the OpenAPI path to the same service layer the web link uses:

  • mapOpenAPIEvalTargetType: add cases a2a_agent -> A2AAgent, custom_agent -> CustomAgent.
  • supportedOpenAPIEvalTargetTypes: add both so the up-front validation and the error message (the
    supported-types list) stay in sync.
  • Unit tests for the two new mappings, plus an end-to-end convertor test covering custom_agent (cluster +
    agent_connection passthrough) and a2a_agent (source_target_id reference).

No IDL change (the OpenAPI string enums and EvalTargetContent sub-fields already exist) and no operator
change (already registered downstream). faas_http (an AccessProtocol, not an EvalTargetType) is still
correctly rejected.

Verified on a PPE swimlane: a2a_agent / custom_agent experiments are created successfully (with field
mappings), and faas_http is rejected with the supported-types list now containing all 8 types.

zh(optional):
网页端已支持 A2A Agent(a2a_agent,type=9)和内场长链接
Agent(custom_agent,type=10),执行算子也已在下游注册,但 OpenAPI
创建实验链路一直拒绝这两类——mapOpenAPIEvalTargetType 只处理 6 类,a2a/custom 落入 default 被拦。本 PR
在转换层补两个 case + 同步白名单,让 OpenAPI 链路接到与网页端相同的服务层。不改 IDL、不改算子;faas_http
仍正确被拒。已在 PPE 泳道验证:a2a/custom 实验创建成功,faas_http 报错且枚举列表已含 8 类。

…es on OpenAPI experiment-create path

The web link already supports A2A Agent (a2a_agent, type=9) and the in-house
long-connection agent (custom_agent, type=10), and the execution operators are
registered downstream. The OpenAPI experiment-create path, however, rejected
these two types: mapOpenAPIEvalTargetType only cased 6 types, so a2a_agent /
custom_agent fell through to default and were blocked both by the up-front
SubmitExperimentOApi validation and by the convertor.

This wires the OpenAPI path to the same service layer the web link uses:
- mapOpenAPIEvalTargetType: add cases a2a_agent -> A2AAgent, custom_agent -> CustomAgent.
- supportedOpenAPIEvalTargetTypes: add both so the up-front validation and the
  error message stay in sync.

No IDL change (the OpenAPI string enums and EvalTargetContent sub-fields already
exist) and no operator change (already registered in the commercial DI).

Meego: 7337555141

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jamesonics
jamesonics requested review from VinCinx and xueyizheng June 24, 2026 10:03
@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #560   +/-   ##
=======================================
  Coverage   77.81%   77.82%           
=======================================
  Files         670      670           
  Lines       76100    76119   +19     
=======================================
+ Hits        59217    59238   +21     
+ Misses      13421    13420    -1     
+ Partials     3462     3461    -1     
Flag Coverage Δ
unittests 77.82% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...uation/application/convertor/experiment/openapi.go 83.64% <100.00%> (+0.03%) ⬆️

... and 11 files with indirect coverage changes


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2011284...dc5a59e. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@jamesonics
jamesonics merged commit 50b9bd9 into main Jun 24, 2026
17 checks passed
@jamesonics
jamesonics deleted the feat/openapi-eval-target-a2a-custom branch June 24, 2026 11:24
Colin4k1024 pushed a commit to Colin4k1024/coze-loop that referenced this pull request Aug 28, 2026
…es on the OpenAPI experiment-create path (coze-dev#560)

[feat][evaluation] support a2a_agent and custom_agent eval target types on OpenAPI experiment-create path

The web link already supports A2A Agent (a2a_agent, type=9) and the in-house
long-connection agent (custom_agent, type=10), and the execution operators are
registered downstream. The OpenAPI experiment-create path, however, rejected
these two types: mapOpenAPIEvalTargetType only cased 6 types, so a2a_agent /
custom_agent fell through to default and were blocked both by the up-front
SubmitExperimentOApi validation and by the convertor.

This wires the OpenAPI path to the same service layer the web link uses:
- mapOpenAPIEvalTargetType: add cases a2a_agent -> A2AAgent, custom_agent -> CustomAgent.
- supportedOpenAPIEvalTargetTypes: add both so the up-front validation and the
  error message stay in sync.

No IDL change (the OpenAPI string enums and EvalTargetContent sub-fields already
exist) and no operator change (already registered in the commercial DI).

Meego: 7337555141

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants