Skip to content

fix(anthropic): support built-in server tools and improve count_tokens response#106

Open
juslintek wants to merge 6 commits intojwadow:mainfrom
juslintek:fix/server-tools-and-count-tokens
Open

fix(anthropic): support built-in server tools and improve count_tokens response#106
juslintek wants to merge 6 commits intojwadow:mainfrom
juslintek:fix/server-tools-and-count-tokens

Conversation

@juslintek
Copy link
Copy Markdown

@juslintek juslintek commented Mar 20, 2026

Problems

1. Built-in server tools cause 422 errors (25 occurrences in logs)

Anthropic's built-in server tools (web_search_20250305, code_execution_20250522, bash_20241022, text_editor_20250124, etc.) don't have input_schema — they use a type field instead. The gateway's AnthropicTool model required input_schema, causing:

Validation error (422): 'input_schema' Field required
input: {'type': 'web_search_20250305', 'name': 'web_search', 'max_uses': 8}

2. count_tokens response missing context_management

The Anthropic API returns {"input_tokens": N, "context_management": {"original_input_tokens": N}} but the gateway only returned {"input_tokens": N}. Clients expecting the full response format may break.

Solution

  • Make AnthropicTool.input_schema optional and add type field with extra="allow" to accept any built-in tool fields
  • Silently strip server tools (no input_schema) in the converter since Kiro API can't handle them — custom tools work as before
  • Add context_management.original_input_tokens to count_tokens response

Backward compatible

  • Custom tools with input_schema work identically
  • Messages endpoint unaffected
  • Only adds acceptance of previously-rejected requests

Depends on

PancakeZik and others added 5 commits March 3, 2026 18:18
@PancakeZik @juslintek
fix: use full Kiro payload for prompt token counting
4d45b2f 
The prompt_tokens reported to clients (used by Claude Code's /context
command) were wildly inaccurate because they were derived from Kiro's
contextUsagePercentage, which returns unreliable values.

Instead, count tokens from the complete serialized Kiro request payload
using tiktoken. This includes system prompt, messages, tools, and all
other payload fields — matching what actually gets sent to the API.

- Replace request_messages/request_tools params with pre-counted
  prompt_tokens across all streaming functions
- Count tokens from full kiro_request_body in both OpenAI and
  Anthropic route handlers
- Remove dependency on contextUsagePercentage for token counting
- Update tests to match new function signatures
@yzddmr6 @claude @juslintek
feat: add /v1/messages/count_tokens endpoint for Anthropic API
5587e2f 
Claude Code calls this endpoint before each request to check conversation
size and decide whether to trigger compaction. Without it, the gateway
returns 404, Claude Code cannot estimate context usage, and long
conversations eventually hit the upstream CONTENT_LENGTH_EXCEEDS_THRESHOLD
error (400).

The endpoint builds the full Kiro payload and counts tokens on the
serialized JSON using tiktoken, consistent with the token counting
approach used in the messages endpoint.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@juslintek
fix(gateway): truncate long tool names instead of rejecting them
d4c9a3a 
Replace validate_tool_names (400 error) with deterministic truncation
for tool names exceeding 64-char Kiro API limit. Names are shortened
to 55 chars + '_' + 8-char md5 hash. Mapping is reversed in responses
so clients receive original names.

Fixes MCP plugins with auto-generated names like
mcp__plugin_cloudflare_cloudflare-docs__search_cloudflare_documentation
@juslintek
fix(anthropic): make max_tokens optional with default for count_token…
772262d 
…s endpoint

Claude Code sends /v1/messages/count_tokens without max_tokens since it
only needs token counting, not generation. The required max_tokens field
caused 422 validation errors, breaking context usage tracking and
preventing conversation compaction.
@juslintek
fix(anthropic): support built-in server tools and improve count_token…
9232a39 
…s response

- Make AnthropicTool.input_schema optional to accept Anthropic built-in
  server tools (web_search, code_execution, bash, text_editor) that
  don't have input_schema. These were causing 422 validation errors.
- Silently strip server tools in converter since Kiro API can't handle
  them, while keeping custom tools working as before.
- Add context_management.original_input_tokens to count_tokens response
  to match Anthropic API spec.
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Mar 20, 2026

Thanks for the PR! 🎉

Before merge, we need a one-time CLA confirmation.
It confirms that you have the right to contribute this code and allow the project to use it.

Full CLA text:
https://github.com/jwadow/kiro-gateway/blob/main/CLA.md

Please reply once with:

I have read the CLA and I accept its terms

You need to write once, all further messages from me can be ignored.

@Met4physics
Copy link
Copy Markdown

Met4physics commented Mar 20, 2026

I guess this is not a good solution for problem 1. It just ignore the problem. Kiro IDE can use the web search tool, so there is a way to use it as api.

@juslintek juslintek mentioned this pull request Mar 21, 2026
Open
@juslintek
Copy link
Copy Markdown
Author

juslintek commented Mar 21, 2026

Good point @Met4physics. You're right that Kiro IDE supports web search internally.

The issue is that the Kiro API uses toolSpecification format which requires name, description, and inputSchema. Anthropic's server tools (web_search_20250305, code_execution_20250522, etc.) don't have inputSchema — they're a different kind of tool declaration.

If we pass them as-is, the Kiro API rejects them with a 422. If we fabricate an inputSchema, the Kiro API wouldn't know to invoke its internal web search — it would just see an unknown tool.

Stripping is the safe default to prevent 422 errors. But I agree this could be improved:

  • If the Kiro API exposes its web search as a tool capability in the future, we could map Anthropic's web_search_20250305 → Kiro's equivalent
  • Or we could add a config flag to pass server tools through for users who want to experiment

Happy to iterate on this if there's a known way to trigger Kiro's web search via the API.

@juslintek
fix: update tests for new API signatures and server tools support
da08f6a 
- Fix server tool detection: check 'input_schema is None' not 'not input_schema'
  (empty dict {} is valid for tools with no parameters)
- Update anthropic_to_kiro tests to use .payload on KiroPayloadResult
- Update build_kiro_payload tests to use .payload on KiroPayloadResult
- Replace validate_tool_names tests with truncate_tool_names tests
- Update AnthropicTool tests for optional name/input_schema
- Update max_tokens test for optional default (4096)

All 1412 tests pass.
@cla-bot
Copy link
Copy Markdown

cla-bot bot commented Mar 21, 2026

Thanks for the PR! 🎉

Before merge, we need a one-time CLA confirmation.
It confirms that you have the right to contribute this code and allow the project to use it.

Full CLA text:
https://github.com/jwadow/kiro-gateway/blob/main/CLA.md

Please reply once with:

I have read the CLA and I accept its terms

You need to write once, all further messages from me can be ignored.

@juslintek
Copy link
Copy Markdown
Author

juslintek commented Mar 21, 2026

I have read the CLA and I accept its terms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@juslintek @Met4physics @PancakeZik @yzddmr6