Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Bug Fixes

* **gain:** use weighted savings rate in per-command stats — `AVG(savings_pct)` replaced by `SUM(saved_tokens)/SUM(input_tokens)`, fixing commands like `grep` showing 14.8% instead of 98.6%; column renamed `Avg%` → `Rate` ([#891](https://github.com/rtk-ai/rtk/pull/891)) ([b3ab873](https://github.com/rtk-ai/rtk/commit/b3ab873))
* **diff:** correct truncation overflow count in condense_unified_diff ([#833](https://github.com/rtk-ai/rtk/pull/833)) ([5399f83](https://github.com/rtk-ai/rtk/commit/5399f83))
* **git:** replace vague truncation markers with exact counts in log and grep output ([#833](https://github.com/rtk-ai/rtk/pull/833)) ([185fb97](https://github.com/rtk-ai/rtk/commit/185fb97))

Expand Down
2 changes: 2 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ For the full architecture, component details, and module development patterns, s

Module responsibilities are documented in each folder's `README.md` and each file's `//!` doc header. Browse `src/cmds/*/` to discover available filters.

Supported ecosystems: git/gh/gt, cargo, go/golangci-lint, npm/pnpm/npx, ruff/pytest/pip/mypy, rspec/rubocop/rake, dotnet, playwright/vitest/jest, docker/kubectl/aws.

Comment on lines +78 to +79
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description is centered on gain’s weighted savings calculation, but this change also updates general repository guidance (supported ecosystems list). Please either mention this documentation update in the PR description or move it to a separate doc-only PR to keep the change set focused.

Copilot uses AI. Check for mistakes.
### Proxy Mode

**Purpose**: Execute commands without filtering but track usage for metrics.
Expand Down
2 changes: 1 addition & 1 deletion src/analytics/gain.rs
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ pub fn run(
println!("{}", "─".repeat(table_width));
println!(
"{:>3} {:<cmd_width$} {:>count_width$} {:>saved_width$} {:>6} {:>time_width$} {:<impact_width$}",
"#", "Command", "Count", "Saved", "Avg%", "Time", "Impact",
"#", "Command", "Count", "Saved", "Rate", "Time", "Impact",
cmd_width = cmd_width, count_width = count_width,
saved_width = saved_width, time_width = time_width,
impact_width = impact_width
Expand Down
78 changes: 76 additions & 2 deletions src/core/tracking.rs
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,12 @@ pub struct MonthStats {
pub avg_time_ms: u64,
}

/// Type alias for command statistics tuple: (command, count, saved_tokens, avg_savings_pct, avg_time_ms)
/// Type alias for command statistics tuple: (command, count, saved_tokens, weighted_savings_rate, avg_time_ms)
///
/// # Warning
/// The 4th field is a **weighted** savings rate: `SUM(saved_tokens) * 100.0 / SUM(input_tokens)`.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// The 4th field is a **weighted** savings rate: `SUM(saved_tokens) * 100.0 / SUM(input_tokens)`.
/// The 4th field is a **weighted** savings rate: `SUM(saved_tokens) / SUM(input_tokens) * 100.0 .

Seems more idiomatic to express a percentage

/// Do NOT aggregate this column with `AVG()` — that would produce an unweighted mean that
/// under-weights high-volume commands. Always use `SUM(saved) / SUM(input)` instead.
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning mixes “percentage” and “ratio” formulas: earlier it defines the field as SUM(saved_tokens) * 100.0 / SUM(input_tokens), but the last sentence says to use SUM(saved) / SUM(input) (missing the * 100.0 and column names). Please make the recommended aggregation formula consistent and explicitly in percent form to avoid copy/paste mistakes.

Suggested change
/// under-weights high-volume commands. Always use `SUM(saved) / SUM(input)` instead.
/// under-weights high-volume commands. Always recompute it as
/// `SUM(saved_tokens) * 100.0 / SUM(input_tokens)` instead.

Copilot uses AI. Check for mistakes.
type CommandStats = (String, usize, usize, f64, u64);

impl Tracker {
Expand Down Expand Up @@ -573,7 +578,9 @@ impl Tracker {
) -> Result<Vec<CommandStats>> {
let (project_exact, project_glob) = project_filter_params(project_path); // added
let mut stmt = self.conn.prepare(
"SELECT rtk_cmd, COUNT(*), SUM(saved_tokens), AVG(savings_pct), AVG(exec_time_ms)
"SELECT rtk_cmd, COUNT(*), SUM(saved_tokens),
CASE WHEN SUM(input_tokens) > 0 THEN SUM(saved_tokens) * 100.0 / SUM(input_tokens) ELSE 0.0 END,
AVG(exec_time_ms)
FROM commands
WHERE (?1 IS NULL OR project_path = ?1 OR project_path GLOB ?2)
GROUP BY rtk_cmd
Expand Down Expand Up @@ -1426,4 +1433,71 @@ mod tests {
// but we can verify recovery_rate is between 0 and 100
assert!(summary.recovery_rate >= 0.0 && summary.recovery_rate <= 100.0);
}

// 14. get_by_command uses weighted savings rate, not unweighted average
//
// Regression test for: AVG(savings_pct) gave wrong results when small invocations
// with 0% savings diluted the average of high-volume commands.
//
// Setup: one small command (10% savings) + one large command (95% savings).
// Unweighted avg would be ~52.5%. Weighted rate must be ~95%.
//
// Uses a unique fake project path so project-filtered query returns only our rows,
// avoiding the LIMIT 10 cap on the shared DB.
#[test]
fn test_get_by_command_weighted_savings_rate() {
let tracker = Tracker::new().expect("Failed to create tracker");
let pid = std::process::id();
let cmd_name = format!("weighted_test_{}", pid);
// Unique project path keeps our rows isolated from the shared DB
let fake_project = format!("/tmp/rtk_weighted_test_{}", pid);

// Override project_path by inserting directly via conn
let saved_small = 10_i64; // 100 in - 90 out = 10 saved → 10%
let saved_large = 95_000_i64; // 100_000 in - 5_000 out = 95_000 saved → 95%
tracker
.conn
.execute(
"INSERT INTO commands (timestamp, original_cmd, rtk_cmd, project_path, input_tokens, output_tokens, saved_tokens, savings_pct, exec_time_ms)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)",
rusqlite::params![
chrono::Utc::now().to_rfc3339(),
&cmd_name, &cmd_name, &fake_project,
100_i64, 90_i64, saved_small, 10.0_f64, 5_i64
],
)
.expect("Failed to insert small invocation");
tracker
.conn
.execute(
"INSERT INTO commands (timestamp, original_cmd, rtk_cmd, project_path, input_tokens, output_tokens, saved_tokens, savings_pct, exec_time_ms)
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7, ?8, ?9)",
rusqlite::params![
chrono::Utc::now().to_rfc3339(),
&cmd_name, &cmd_name, &fake_project,
100_000_i64, 5_000_i64, saved_large, 95.0_f64, 10_i64
],
)
.expect("Failed to insert large invocation");

let by_cmd = tracker
.get_by_command(Some(&fake_project))
.expect("Failed to get by_command stats");

let entry = by_cmd
.iter()
.find(|(name, _, _, _, _)| name == &cmd_name)
.expect("Test command not found in by_command stats");

let (_name, _count, _saved, rate, _time) = entry;

// Weighted rate = (10 + 95_000) * 100.0 / (100 + 100_000) ≈ 94.99%
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The regression test comment’s expected weighted value looks off: (10 + 95_000) * 100 / (100 + 100_000) is ~94.9%, not ~94.99%. Consider correcting the comment (or compute the expected rate and assert within a small epsilon) so the test documents the math accurately.

Suggested change
// Weighted rate = (10 + 95_000) * 100.0 / (100 + 100_000) ≈ 94.99%
// Weighted rate = (10 + 95_000) * 100.0 / (100 + 100_000) ≈ 94.9%

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact final value is 94.9150849150849150 so yeah much more near 94.9 than 94.99

// Unweighted avg would be (10.0 + 95.0) / 2 = 52.5%
// The gap proves the fix works.
assert!(
*rate > 90.0,
"Expected weighted rate >90%, got {:.1}% — unweighted avg would be ~52.5%",
rate
);
}
}
133 changes: 129 additions & 4 deletions src/discover/registry.rs
Original file line number Diff line number Diff line change
Expand Up @@ -890,9 +890,8 @@ mod tests {

#[test]
fn test_registry_covers_all_cargo_subcommands() {
// Verify that every CargoCommand variant (Build, Test, Clippy, Check, Fmt)
// except Other has a matching pattern in the registry
for subcmd in ["build", "test", "clippy", "check", "fmt"] {
// Verify that every CargoCommand variant except Other has a matching pattern
for subcmd in ["build", "test", "clippy", "check", "fmt", "run", "publish"] {
let cmd = format!("cargo {subcmd}");
match classify_command(&cmd) {
Classification::Supported { .. } => {}
Expand All @@ -906,7 +905,7 @@ mod tests {
// Verify that every GitCommand subcommand has a matching pattern
for subcmd in [
"status", "log", "diff", "show", "add", "commit", "push", "pull", "branch", "fetch",
"stash", "worktree",
"stash", "worktree", "checkout", "merge", "rebase",
] {
let cmd = format!("git {subcmd}");
match classify_command(&cmd) {
Expand Down Expand Up @@ -2348,4 +2347,130 @@ mod tests {
assert_eq!(strip_git_global_opts("git status"), "git status");
assert_eq!(strip_git_global_opts("cargo test"), "cargo test");
}

// Passthrough subcommand coverage: discover should classify these as Supported
// (routing to passthrough handlers) rather than "TOP UNHANDLED COMMANDS".

#[test]
fn test_classify_git_checkout_passthrough() {
assert_eq!(
classify_command("git checkout main"),
Classification::Supported {
rtk_equivalent: "rtk git",
category: "Git",
estimated_savings_pct: 70.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_classify_git_merge_passthrough() {
assert_eq!(
classify_command("git merge feature/my-branch"),
Classification::Supported {
rtk_equivalent: "rtk git",
category: "Git",
estimated_savings_pct: 70.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_classify_git_rebase_passthrough() {
assert_eq!(
classify_command("git rebase origin/main"),
Classification::Supported {
rtk_equivalent: "rtk git",
category: "Git",
estimated_savings_pct: 70.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_classify_cargo_run_passthrough() {
assert_eq!(
classify_command("cargo run -- verify"),
Classification::Supported {
rtk_equivalent: "rtk cargo",
category: "Cargo",
estimated_savings_pct: 80.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_classify_cargo_publish_passthrough() {
assert_eq!(
classify_command("cargo publish -p my-crate"),
Classification::Supported {
rtk_equivalent: "rtk cargo",
category: "Cargo",
estimated_savings_pct: 80.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_classify_pnpm_build_passthrough() {
assert_eq!(
classify_command("pnpm build"),
Classification::Supported {
rtk_equivalent: "rtk pnpm",
category: "PackageManager",
estimated_savings_pct: 80.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_classify_pnpm_exec_passthrough() {
assert_eq!(
classify_command("pnpm exec eslint --max-warnings=0"),
Classification::Supported {
rtk_equivalent: "rtk pnpm",
category: "PackageManager",
estimated_savings_pct: 80.0,
status: RtkStatus::Passthrough,
}
);
}

#[test]
fn test_rewrite_git_checkout() {
assert_eq!(
rewrite_command("git checkout main", &[]),
Some("rtk git checkout main".to_string())
);
}

#[test]
fn test_rewrite_git_merge() {
assert_eq!(
rewrite_command("git merge --no-edit feature", &[]),
Some("rtk git merge --no-edit feature".to_string())
);
}

#[test]
fn test_rewrite_cargo_run() {
assert_eq!(
rewrite_command("cargo run -- verify", &[]),
Some("rtk cargo run -- verify".to_string())
);
}

#[test]
fn test_rewrite_pnpm_build() {
assert_eq!(
rewrite_command("pnpm build", &[]),
Some("rtk pnpm build".to_string())
);
}
}
23 changes: 17 additions & 6 deletions src/discover/rules.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ pub struct RtkRule {

// Patterns ordered to match RULES indices exactly.
pub const PATTERNS: &[&str] = &[
r"^git\s+(?:-[Cc]\s+\S+\s+)*(status|log|diff|show|add|commit|push|pull|branch|fetch|stash|worktree)",
r"^git\s+(?:-[Cc]\s+\S+\s+)*(status|log|diff|show|add|commit|push|pull|branch|fetch|stash|worktree|checkout|merge|rebase)",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using alphabetically sorted list would ease maintenance

r"^gh\s+(pr|issue|run|repo|api|release)",
r"^cargo\s+(build|test|clippy|check|fmt|install)",
r"^pnpm\s+(list|ls|outdated|install)",
r"^cargo\s+(build|test|clippy|check|fmt|install|run|publish)",
r"^pnpm\s+(list|ls|outdated|install|build|exec)",
Comment on lines +18 to +21
Copy link

Copilot AI Mar 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description focuses on weighted savings rate in gain, but this hunk also expands Discover’s supported command patterns/statuses (git checkout/merge/rebase, cargo run/publish, pnpm build/exec). Please update the PR description to include this scope change (or split into a separate PR) so reviewers can evaluate the intent/risk appropriately.

Copilot uses AI. Check for mistakes.
r"^npm\s+(run|exec)",
r"^npx\s+",
r"^(cat|head|tail)\s+",
Expand Down Expand Up @@ -102,7 +102,11 @@ pub const RULES: &[RtkRule] = &[
("add", 59.0),
("commit", 59.0),
],
subcmd_status: &[],
subcmd_status: &[
("checkout", RtkStatus::Passthrough),
("merge", RtkStatus::Passthrough),
("rebase", RtkStatus::Passthrough),
],
},
RtkRule {
rtk_cmd: "rtk gh",
Expand All @@ -118,15 +122,22 @@ pub const RULES: &[RtkRule] = &[
category: "Cargo",
savings_pct: 80.0,
subcmd_savings: &[("test", 90.0), ("check", 80.0)],
subcmd_status: &[("fmt", RtkStatus::Passthrough)],
subcmd_status: &[
("fmt", RtkStatus::Passthrough),
("run", RtkStatus::Passthrough),
("publish", RtkStatus::Passthrough),
],
},
RtkRule {
rtk_cmd: "rtk pnpm",
rewrite_prefixes: &["pnpm"],
category: "PackageManager",
savings_pct: 80.0,
subcmd_savings: &[],
subcmd_status: &[],
subcmd_status: &[
("build", RtkStatus::Passthrough),
("exec", RtkStatus::Passthrough),
],
},
RtkRule {
rtk_cmd: "rtk npm",
Expand Down
Loading