Repository navigation
fix(llm): carry the cache-write TTL split from Anthropic message_start usage - #4
Merged
danielzarioiu merged 1 commit intoOct 10, 2026
Conversation
…t usage
RC2 production canary proved the terminal message_delta usage carries NO
cache_creation split on the real Anthropic streaming wire — the CLI-visible
split (production probe: aggregate 67759 with {5m: 67759, 1h: 0}) is reported
at message_start, where input-side usage is final. RC2's delta-only read
therefore still left streaming writes TTL-ambiguous.
- passthrough + translation MessageStart handlers now forward
usage.cache_creation into evidence; the terminal MessageDelta still wins
when IT carries a split, and no longer clobbers start evidence when it
does not.
- Evidence-safety note replacing the earlier consistency-guard idea:
captured production usage shows a small aggregate alongside an all-zero
split (stream_basic: aggregate 13, thinking: aggregate 1). That shape is
the provider's authoritative sub-minimum no-op write evidence — the
classes are authoritative and the residual aggregate meters at zero.
Gating it to unknown would flood cache_write_ttl_ambiguous fail-closed
noise for every sub-minimum cache_control marker; forwarding it matches
what the provider actually bills. The four anthropic streaming goldens
gain exactly the two evidence fields with 0 values (verified per-diff).
- buffered cache_ttl_split keeps bc74277 semantics (no guard).
agent-llm 427/427 (423 + 4), agentgateway 2229/2229, fmt + clippy clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #3, driven by the RC2 production canary: the terminal message_delta usage carries NO cache_creation split on the real Anthropic streaming wire — the split is reported at message_start (input-side usage final; production probe aggregate 67759 with {5m: 67759, 1h: 0}).
Forwards the split from message_start in both streaming converters; the terminal delta wins when it carries a split and no longer clobbers start evidence when it does not. Evidence-safety: captured production usage shows a small aggregate with an all-zero split (sub-minimum no-op cache writes) — forwarded as authoritative Some(0)/Some(0) rather than gated to unknown, which would flood cache_write_ttl_ambiguous fail-closed noise for every sub-minimum cache_control marker; matches what the provider actually bills. Buffered semantics unchanged (bc74277). Goldens gain exactly the two evidence fields (per-diff verified). agent-llm 427/427, agentgateway 2229/2229, fmt+clippy clean.