@tanstack/ai-openrouter: structuredOutput() drops response.usage (cost/tokens never reach consumers)

[tombeckenham]

Summary

OpenRouterTextAdapter.structuredOutput() receives a full chat completion from the SDK (including response.usage with token counts and OpenRouter’s per-request cost), but returns only { data, rawText }. The optional usage field on StructuredOutputResult is never set.

Downstream:

• chat({ outputSchema, stream: false }) → agentic/final structured path → consumers/middleware onFinish / onUsage see no usage
• fallbackStructuredOutputStream (core) now forwards result.usage when present (fallbackStructuredOutputStream drops usage from structured-output.complete and RUN_FINISHED chunks #758), but OpenRouter never populates it, so RUN_FINISHED.usage stays missing on that path too
• Billing/observability layers that rely on TokenUsage.cost (or even token counts) charge $0 or report missing cost after a successful structured call

This is not the same as #758 (engine dropped adapter usage). That was fixed. Here the adapter never returns usage.

Environment

• @tanstack/ai@0.42.0
• @tanstack/ai-openrouter@0.15.10
• Non-stream structured call: chat({ adapter, outputSchema, stream: false, ... }) (or any path that ends in adapter.structuredOutput())

Reproduction

import { chat } from '@tanstack/ai'
import { openRouterText } from '@tanstack/ai-openrouter'
import { z } from 'zod'

const schema = z.object({ title: z.string() })

// Capture middleware
let usageFromFinish: unknown
const middleware = [
  {
    onFinish(_ctx: unknown, info: { usage?: unknown }) {
      usageFromFinish = info.usage
    },
    onUsage(_ctx: unknown, usage: unknown) {
      console.log('onUsage', usage)
    },
  },
]

const result = await chat({
  adapter: openRouterText('openai/gpt-4o-mini', { apiKey: process.env.OPENROUTER_API_KEY! }),
  messages: [{ role: 'user', content: 'Return a short title as JSON.' }],
  outputSchema: schema,
  stream: false,
  middleware,
})

console.log('result', result)
console.log('onFinish usage', usageFromFinish) // undefined / no cost

Also verify inside the adapter (or via network): the raw OpenRouter response includes usage.prompt_tokens, usage.completion_tokens, and often usage.cost, while the return value of structuredOutput does not.

Root cause

In @tanstack/ai-openrouter adapters/text (structuredOutput):

const response = await this.orClient.chat.send({ chatRequest: { ... stream: false, responseFormat: { type: 'json_schema', ... } } })
// ...
return {
  data: transformed,
  rawText,
  // response.usage is never mapped onto StructuredOutputResult.usage
}

Contrast with streaming helpers in the same adapter, which do:

const finalUsage = buildOpenRouterUsage(lastUsage)
yield {
  type: EventType.RUN_FINISHED,
  ...(finalUsage && {
    usage: { ...finalUsage, ...extractUsageCost(lastUsage) },
  }),
}

StructuredOutputResult already documents the field:

export interface StructuredOutputResult<T = unknown> {
  data: T
  rawText: string
  usage?: TokenUsage  // optional, if provided by the adapter
}

Suggested fix

In OpenRouterTextAdapter.structuredOutput (and any other provider structuredOutput that omits it):

 const transformed = this.transformStructuredOutput(parsed)
+const baseUsage = buildOpenRouterUsage(response.usage)
 return {
   data: transformed,
   rawText,
+  ...(baseUsage && {
+    usage: { ...baseUsage, ...extractUsageCost(response.usage) },
+  }),
 }

That matches the stream path and the contract used by fallbackStructuredOutputStream after #758.

Worth auditing other providers’ structuredOutput() for the same drop.

Impact

Any app that bills or traces from middleware usage after chat({ outputSchema, stream: false }) (or non-stream agentic structured finalization) sees successful completions with no token/cost signal. Streaming structured output can still work when RUN_FINISHED.usage is set; the gap is specifically the non-stream structuredOutput return.

Related

• fallbackStructuredOutputStream drops usage from structured-output.complete and RUN_FINISHED chunks #758 (closed) — core fallback dropped result.usage (fixed); does not cover adapter not returning usage
• chat({ outputSchema }) runs final structured output request outside middleware #390 (closed) — structured final call / middleware coverage
• Add a required billingUnit field to TokenUsage so billed quantities are self-describing #816 (open) — billingUnit on TokenUsage (orthogonal shape work)

Expected

StructuredOutputResult.usage is populated from response.usage (tokens + OpenRouter cost / costDetails when present), and is available to middleware onFinish / onUsage and to any stream fallback that forwards result.usage.