Skip to content

feat(cli): add --source filter to JSON outputs#221

Open
YESHYUNGSEOK wants to merge 2 commits into
mag123c:mainfrom
YESHYUNGSEOK:feat/cli-source-filter
Open

feat(cli): add --source filter to JSON outputs#221
YESHYUNGSEOK wants to merge 2 commits into
mag123c:mainfrom
YESHYUNGSEOK:feat/cli-source-filter

Conversation

@YESHYUNGSEOK

Copy link
Copy Markdown

Summary

Adds a --source <SOURCE> flag to the four JSON commands (daily, weekly, monthly, stats) so scripts can read a single CLI's usage instead of the merged-all-sources series.

toktrack daily --json --source codex
toktrack monthly --json --source claude-code

Motivation

The merged --json output is great for a personal overview, but external tooling (cost dashboards, team token leaderboards) needs per-CLI numbers. Today the only workarounds are re-splitting the merged output by model-name heuristics (fragile — Claude model keys carry no ::provider suffix) or reading the internal cache files directly (undocumented format). The data is already there: the loader builds unmerged per-source summaries (LoadResult.source_summaries, already consumed by report), so this PR only exposes it at the CLI layer — no aggregation changes.

Changes

  • Daily/Weekly/Monthly/Stats accept --source <SOURCE> (clap requires = "json", so TUI behavior is untouched)
  • New select_summaries() picks one source's summaries from LoadResult, or the merged view when the flag is absent — output without the flag is byte-identical to before
  • Ids match toktrack audit (e.g. claude-code, codex, and remote ids like codex@host); unknown ids fail with the sorted list of available ids (ToktrackError::UnknownSource)
  • README: two usage examples

Tests

  • cargo fmt --check / cargo clippy --all-targets --all-features -- -D warnings / cargo test all pass (1,316 tests + 1 doctest, 0 failures)
  • New: parse tests for --source (incl. requires = "json" across all four commands), select_summaries unit tests (merged default / single source / unknown id error message)

Manual verification (real data)

  • daily --json --source codex matches the per-source cache values digit-for-digit; --source claude-code returns the disjoint series
  • unknown id → exit 1 with unknown source: 'x' (available: antigravity, claude-code, codex, ...)
  • --source without --json → standard clap error (exit 2)
  • no flag → unchanged merged output

🤖 Generated with Claude Code

daily/weekly/monthly/stats --json merge every detected CLI into one
series, so external tooling (cost dashboards, team leaderboards)
cannot read a single CLI's usage without re-splitting the merged
output by model-name heuristics.

The loader already builds unmerged per-source summaries
(LoadResult.source_summaries, consumed by `report`); expose them
behind a `--source <SOURCE>` flag on the four JSON commands. Ids
match `toktrack audit` (e.g. claude-code, codex, codex@remote).
Unknown ids fail listing the available ids; the flag requires
--json; output without the flag is unchanged.

Co-authored-by: Claude <noreply@anthropic.com>

@mag123c mag123c left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this — the design is right where I'd want it. Reusing the existing LoadResult.source_summaries instead of adding an aggregation path, requires = "json" so the TUI surface is untouched, and an error message that lists the valid ids rather than just failing: all good calls. I verified the no-flag compatibility claim by diffing main against this branch across 206 days of real data — per-date totals are identical, so the merged path is genuinely unchanged.

One blocking issue before merge.

Zero-data sources behave differently depending on cache state

A source that is registered but has produced no entries errors on a cold cache and returns [] on a warm one. Same command, back to back on the same machine:

### RUN 1 (cold, no cache yet)
$ toktrack daily --json --source qwen
Error: unknown source: 'qwen' (available: claude-code, codex, gemini)
exit=1

### RUN 2 (identical command, cache now warm)
$ toktrack daily --json --source qwen
[]
exit=0

Cause: load_cold_path skips empty sources before they are recorded:

// src/services/data_loader.rs:280
if entries.is_empty() {
    continue;   // source id never inserted into source_summaries
}

load_warm_path has no equivalent guard — it always calls cache_service.load_or_compute, so the id lands in source_summaries with an empty vec. select_summaries is the first consumer that turns a missing key into a hard error; report (src/report/data.rs:133) only iterates, so the asymmetry was invisible until now.

Why it matters here: all eight local parsers are registered unconditionally (src/parsers/mod.rs:164-172), and run_audit lists them straight from registry.sources(). So toktrack audit prints copilot (no data) and qwen (no data) on a typical machine, while --source copilot on a cold cache reports those same ids as unknown. The flag's help text promises "id as shown by toktrack audit", which is the contract I'd like to keep. The cold path is also exactly the first-install / CI / post-cache-version-bump case — the scripting scenario this flag exists to serve.

Suggested fix — seed the key before the empty check in load_cold_path:

source_summaries.entry(source.id.clone()).or_default();

if entries.is_empty() {
    continue;
}

This leaves all_summaries, any_entries, source_stats and source_usage untouched, so merged output is unaffected. A source that fails to parse stays absent on both paths, which is symmetric and fine as-is — only the empty case needs fixing.

Test: the four select_summaries tests are solid (each pins a distinct total, and the error test pins both the id and the sorted list), but they build LoadResult by hand, so none of them cover this. A loader-level test — one source yielding entries, one yielding zero, asserting both ids appear in source_summaries — would pin the behavior and catch a regression if the guard ever moves back.

Happy to merge once that's in.

@YESHYUNGSEOK

Copy link
Copy Markdown
Author

Fixed in 4325ee3 — seeded the key before the empty check in load_cold_path as suggested, so an empty-but-registered source now returns [] on both cache paths instead of erroring only on a cold cache. Added a loader-level test (test_cold_path_records_zero_data_source) with one source yielding entries and one yielding zero, asserting both ids land in source_summaries. Parse failures stay absent on both paths, as you noted.

@YESHYUNGSEOK
YESHYUNGSEOK requested a review from mag123c July 22, 2026 00:36
@mag123c

mag123c commented Jul 22, 2026

Copy link
Copy Markdown
Owner

I think the push didn't land — 4325ee3 isn't on the remote and the PR head is still 068843c. Mind pushing again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants