[feat][evaluation] support a2a_agent and custom_agent eval target types on the OpenAPI experiment-create path - #560
Merged
Conversation
…es on OpenAPI experiment-create path The web link already supports A2A Agent (a2a_agent, type=9) and the in-house long-connection agent (custom_agent, type=10), and the execution operators are registered downstream. The OpenAPI experiment-create path, however, rejected these two types: mapOpenAPIEvalTargetType only cased 6 types, so a2a_agent / custom_agent fell through to default and were blocked both by the up-front SubmitExperimentOApi validation and by the convertor. This wires the OpenAPI path to the same service layer the web link uses: - mapOpenAPIEvalTargetType: add cases a2a_agent -> A2AAgent, custom_agent -> CustomAgent. - supportedOpenAPIEvalTargetTypes: add both so the up-front validation and the error message stay in sync. No IDL change (the OpenAPI string enums and EvalTargetContent sub-fields already exist) and no operator change (already registered in the commercial DI). Meego: 7337555141 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
xueyizheng
approved these changes
Jun 24, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. @@ Coverage Diff @@
## main #560 +/- ##
=======================================
Coverage 77.81% 77.82%
=======================================
Files 670 670
Lines 76100 76119 +19
=======================================
+ Hits 59217 59238 +21
+ Misses 13421 13420 -1
+ Partials 3462 3461 -1
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 11 files with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
VinCinx
approved these changes
Jun 24, 2026
Colin4k1024
pushed a commit
to Colin4k1024/coze-loop
that referenced
this pull request
Aug 28, 2026
…es on the OpenAPI experiment-create path (coze-dev#560) [feat][evaluation] support a2a_agent and custom_agent eval target types on OpenAPI experiment-create path The web link already supports A2A Agent (a2a_agent, type=9) and the in-house long-connection agent (custom_agent, type=10), and the execution operators are registered downstream. The OpenAPI experiment-create path, however, rejected these two types: mapOpenAPIEvalTargetType only cased 6 types, so a2a_agent / custom_agent fell through to default and were blocked both by the up-front SubmitExperimentOApi validation and by the convertor. This wires the OpenAPI path to the same service layer the web link uses: - mapOpenAPIEvalTargetType: add cases a2a_agent -> A2AAgent, custom_agent -> CustomAgent. - supportedOpenAPIEvalTargetTypes: add both so the up-front validation and the error message stay in sync. No IDL change (the OpenAPI string enums and EvalTargetContent sub-fields already exist) and no operator change (already registered in the commercial DI). Meego: 7337555141 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
feat
Check the PR title
(Optional) Translate the PR title into Chinese
[feat][evaluation] OpenAPI 创建实验链路支持 a2a_agent 与 custom_agent 评测对象类型
(Optional) More detailed description for this PR
en:
The web link already supports two eval target types — A2A Agent (
a2a_agent, type=9) and the in-houselong-connection agent (
custom_agent, type=10) — and their execution operators are registered downstream. TheOpenAPI experiment-create path, however, rejected them:
mapOpenAPIEvalTargetTypeonly handled 6 types, soa2a_agent/custom_agentfell through todefaultand were blocked both by the up-frontSubmitExperimentOApivalidation and by the convertor.This PR wires the OpenAPI path to the same service layer the web link uses:
mapOpenAPIEvalTargetType: add casesa2a_agent -> A2AAgent,custom_agent -> CustomAgent.supportedOpenAPIEvalTargetTypes: add both so the up-front validation and the error message (thesupported-types list) stay in sync.
custom_agent(cluster +agent_connection passthrough) and
a2a_agent(source_target_id reference).No IDL change (the OpenAPI string enums and
EvalTargetContentsub-fields already exist) and no operatorchange (already registered downstream).
faas_http(an AccessProtocol, not an EvalTargetType) is stillcorrectly rejected.
Verified on a PPE swimlane:
a2a_agent/custom_agentexperiments are created successfully (with fieldmappings), and
faas_httpis rejected with the supported-types list now containing all 8 types.zh(optional):
网页端已支持 A2A Agent(
a2a_agent,type=9)和内场长链接Agent(
custom_agent,type=10),执行算子也已在下游注册,但 OpenAPI创建实验链路一直拒绝这两类——
mapOpenAPIEvalTargetType只处理 6 类,a2a/custom 落入 default 被拦。本 PR在转换层补两个 case + 同步白名单,让 OpenAPI 链路接到与网页端相同的服务层。不改 IDL、不改算子;faas_http
仍正确被拒。已在 PPE 泳道验证:a2a/custom 实验创建成功,faas_http 报错且枚举列表已含 8 类。