feat: Add prompt caching to OpenAI-compatible custom models

[dleen]

Context

Builds on the PR: #1562 to add the OpenAI compatible provider support for cache control. The previous PR updates the UI to add an option to specify that a model supports prompt caching.

Implementation

The OpenRouter provider has an implementation of adding the cache control key to OpenAI messages. Acknowledging the risk of duplication we pretty much copy the implementation wholesale. There probably is an opportunity to combine the implementations for OpenAI compatible and OpenRouter in the future.

Screenshots

How to Test

1. Selected the prompt caching option in the UI. Entered gateway server for base URL.
2. Started a chat request.
3. On the gateway server observed the cache control key in the messages: 'type': 'text', 'cache_control': {'type': 'ephemeral'}
4. On the gateway server observe the usage response:

'usage': {'cacheReadInputTokenCount': 16974, 'cacheReadInputTokens': 16974, 'cacheWriteInputTokenCount': 4438, 'cacheWriteInputTokens': 4438, 'inputTokens': 4, 'outputTokens': 222, 'totalTokens': 21638}

Get in Touch

Roo Code Discord handle: dleen

---

Important

Adds prompt caching support for OpenAI-compatible custom models, including UI options and message handling in openai.ts.

• Behavior:
  • Adds prompt caching support to OpenAI-compatible custom models in openai.ts.
  • Implements cache control in createMessage() for models with supportsPromptCache.
  • Copies logic from OpenRouter to add cache_control to user messages.
• UI Changes:
  • Adds "Prompt Caching" checkbox in ApiOptions.tsx for OpenAI-compatible models.
  • Allows configuration of cache read/write prices when prompt caching is enabled.
• Misc:
  • Updates .changeset/thin-fans-deliver.md to document the feature addition.

^{This description was created by for 10c1e7d. It will automatically update as commits are pushed.}

[changeset-bot]

🦋 Changeset detected

Latest commit: 10c1e7d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
roo-cline	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[ellipsis-dev]

Avoid using // @ts-ignore-next-line to bypass type errors for the cache_control property. Consider extending the type definitions instead, and also evaluate extracting this caching logic into a shared helper to reduce duplication with the OpenRouter implementation.

[mrubens]

Thank you!