Skip to content

Update benchmark_serving.py#1705

Open
puneithk wants to merge 1 commit into
SemiAnalysisAI:mainfrom
puneithk:patch-1
Open

Update benchmark_serving.py#1705
puneithk wants to merge 1 commit into
SemiAnalysisAI:mainfrom
puneithk:patch-1

Conversation

@puneithk

@puneithk puneithk commented Jun 10, 2026

Copy link
Copy Markdown

We need to have input and output token length for performance tracking.


Note

Low Risk
Single change to which JSON fields are dropped on save; no runtime serving or benchmark execution behavior changes.

Overview
Keeps per-request token lengths in default benchmark JSON exports so performance tracking can use them without turning on full detailed dumps.

When --save-result is used and --save-detailed is not, the saver still strips high-volume fields (ttfts, itls, generated_texts, errors) to limit file size, but it no longer removes input_lens and output_lens from the merged result.

Reviewed by Cursor Bugbot for commit 7a260a1. Bugbot is set up for automated code reviews on this repo. Configure here.

We need to have input and output token length for performance tracking.
@puneithk puneithk requested a review from a team June 10, 2026 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant