Skip to content

Close fully redacted tool proposals - #1149

Open
KeenWill wants to merge 1 commit into
agent/daemon-live-eligibility-latest-goal-scanfrom
agent/daemon-live-redacted-tool-closeout-pr
Open

Close fully redacted tool proposals#1149
KeenWill wants to merge 1 commit into
agent/daemon-live-eligibility-latest-goal-scanfrom
agent/daemon-live-redacted-tool-closeout-pr

Conversation

@KeenWill

Copy link
Copy Markdown
Owner

Summary

  • represent fully credential-redacted tool proposals as typed suppressed assistant material
  • prevent suppressed proposals from entering the executable tool-request loop
  • classify a response containing only suppressed tool material as known-failed with unrepresentable_tool_material
  • preserve mixed responses: ordinary assistant text remains usable while the suppressed tool call is omitted

Meaningfully changed lines: 314 (excluding Cargo.lock).

Numeric-bound ceilings added: none.

Validation

Focused Codex, Claude, and provider-runtime tests pass. Workspace check, warning-denied Clippy, doctests, warning-denied rustdoc, metadata, and the required PostgreSQL integration suite pass.

Copilot AI lite review requested due to automatic review settings August 22, 2026 23:48
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for security reviews. Please try again later.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Warning

Your free Security trial is over. An organization admin can activate billing to continue.

@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: b79e1070-33a4-4e58-85a4-12ed9840aabe

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 576d1dd0f8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +400 to +401
#[tokio::test]
async fn fully_suppressed_tool_arguments_are_non_executable() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Tag this credential-boundary test with INV-035

This scenario directly enforces the INV-035 credential boundary—whole-object credential suppression must not emit executable tool material—but neither its name nor its doc comment identifies that invariant, unlike the surrounding credential-boundary tests. Add the INV-035 tag so invariant-focused discovery can find this new enforcement.

AGENTS.md reference: AGENTS.md:L231-L235

Useful? React with 👍 / 👎.

Comment on lines +2985 to +2989
/// A CLI-redacted argument object is not an executable tool request. The
/// completed provider call closes as the existing unrepresentable-material
/// failure instead of entering the tool loop with sentinel JSON.
#[test]
fn fully_suppressed_tool_arguments_close_as_known_failure() {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Tag this terminal-closeout test with INV-014

This test establishes the INV-014 terminal-closeout behavior for a newly introduced class of unrepresentable tool material, yet neither the name nor the attached comment carries the invariant tag. Add INV-014 so the test is discoverable alongside the adjacent malformed-proposal closeout coverage.

AGENTS.md reference: AGENTS.md:L231-L235

Useful? React with 👍 / 👎.

Comment on lines +109 to +111
/// A tool call whose argument object the CLI credential boundary suppressed
/// as a whole. No executable proposal exists beyond the adapter boundary.
SuppressedToolCall,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Reject suppressed parts during structured decoding

When a structured completion contains one valid contract proposal plus an additional suppressed proposal, decode_structured ignores this new variant and sees exactly one matching value. RuntimeApprovalJudgeModel calls that decoder directly rather than going through classify_terminal, so it can accept an approval decision from a response that also contained an extra hidden tool call; before this change, the sentinel remained a ToolCall and was counted as a multiplicity failure. Treat any suppressed part as an explicit structured-decode failure.

Useful? React with 👍 / 👎.

Comment on lines +1271 to +1273
a tool argument object suppressed as a whole crosses the adapter as typed
non-executable material and closes the model call as a known failure, never as
sentinel JSON that could enter the tool loop; a private-key PEM block is

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Document Claude's suppressed-tool closeout

The implementation now emits SuppressedToolCall from both CLI adapters, but this added contract remains inside the Codex-only credential paragraph and the verification note explicitly names only Codex. Consequently the owning runtime specification does not describe the new Claude behavior even though model-runtime-claude-cli/src/event.rs implements it; extend this section or the Claude credential section to cover that adapter as well.

AGENTS.md reference: AGENTS.md:L46-L48

Useful? React with 👍 / 👎.

@github-actions

Copy link
Copy Markdown
Contributor

Rust coverage (report only)

Report only. This measurement has no threshold, gates no merge, and
fails no check; it exists so untested code stays visible.

Measured suite Outcome
workspace (--all-targets --all-features) success
persistence PostgreSQL (--ignored) success
signalboxd PostgreSQL (--ignored) failure
terminal-client PostgreSQL (--ignored) success
What this number does not measure
  • Doctests. cargo llvm-cov --doctests needs a nightly
    toolchain; this workspace pins stable, so the compile-fail
    sealing proofs and every other doctest are outside the
    denominator.
  • Live smokes, which spend real network requests or
    credentials and are never run here: the tools-github and
    tools-web smokes stay --ignored, and the whole-daemon and
    real-provider terminal smokes are skipped by name above.
  • The Swift native client, which Xcode measures separately.
  • One environment-clearing test in signalbox-tools-exec,
    which instrumenting its supervisor process perturbs. The
    workflow comment on the workspace step states why; rust.yml
    runs that test uninstrumented and gates on it.
  • Dedicated test files. cargo-llvm-cov excludes tests/
    and benches/ targets and *tests.rs modules from the
    report by default, so that test code is counted neither
    covered nor uncovered here. Inline #[cfg(test)] modules
    inside a source file are the exception: they are
    instrumented, and they land on both sides. A test body
    that ran counts as covered, which makes every percentage
    below optimistic; the body of an #[ignore]d test no
    measured suite runs counts as uncovered, which puts test
    lines into the file ranking. Read both tables as close,
    not exact.
Measure Covered Total Percent
Lines 252199 308770 81.68%
Functions 20531 24922 82.38%
Regions 319799 399804 79.99%

Per crate, least-covered first

Crate Line % Lines Function % Region %
crates/program-runtime 7.57% 38/502 5.77% 6.13%
crates/runner-wire 60.41% 644/1066 62.67% 59.14%
apps/signalboxd 63.87% 34894/54634 71.15% 63.73%
crates/tools-sessions 65.06% 378/581 63.29% 61.68%
apps/signalbox-runner 69.47% 1784/2568 70.93% 70.29%
crates/approval-judge-eval 73.87% 492/666 76.12% 76.41%
crates/tool-schema-derive 74.20% 279/376 88.00% 72.42%
crates/tools-basic 75.37% 771/1023 66.15% 79.16%
crates/model-runtime-claude-cli 76.39% 1644/2152 72.96% 77.25%
crates/tools-github 77.06% 2408/3125 74.19% 74.82%
crates/tools-exec 77.66% 5189/6682 74.88% 73.93%
apps/client 78.66% 13026/16560 89.37% 75.61%
crates/persistence 80.03% 53812/67236 77.98% 75.44%
crates/tools-code-host 80.33% 6713/8357 81.17% 76.66%
crates/blob-store-filesystem 80.50% 1371/1703 70.17% 80.45%
crates/model-provider-runtime 81.36% 2431/2988 85.65% 79.91%
crates/blob-store 82.31% 307/373 75.41% 83.49%
crates/tools-conversations 83.01% 508/612 83.78% 77.65%
crates/conversation-import-claude-code 85.45% 740/866 66.09% 84.67%
crates/tools-plan 85.71% 768/896 88.89% 81.74%
crates/conversation-import-codex 85.76% 873/1018 64.00% 83.56%
crates/tools-workspace 86.36% 2977/3447 82.14% 87.30%
crates/tools-web 86.59% 2977/3438 87.93% 85.03%
crates/tools-git 87.40% 8750/10011 86.50% 83.24%
crates/egress-transport 88.51% 131/148 88.24% 79.18%
crates/application 88.55% 18929/21376 87.75% 88.75%
crates/model-runtime-codex-cli 89.97% 1543/1715 90.91% 89.91%
crates/test-bin 90.91% 10/11 100.00% 70.00%
crates/domain 92.28% 60105/65135 91.38% 94.13%
crates/web-contract 92.31% 408/442 82.61% 82.09%
crates/conversation-import-json 92.51% 284/307 96.88% 91.36%
crates/model-runtime-openai 93.49% 3806/4071 97.14% 91.73%
crates/process-protocol 93.64% 8485/9061 95.72% 86.75%
crates/model-runtime-anthropic 93.82% 3904/4161 97.15% 91.27%
crates/model-runtime 94.06% 9155/9733 93.29% 94.78%
crates/expect-table 95.60% 977/1022 100.00% 95.75%
crates/tool-contract 97.18% 688/708 96.47% 95.54%

25 files with the most uncovered lines

File Uncovered lines Line %
apps/signalboxd/src/process_runtime.rs 6541 53.08%
apps/signalboxd/src/repo_watch_runtime.rs 2230 69.19%
apps/signalboxd/src/runner_protocol_runtime.rs 2209 34.78%
crates/persistence/src/submit_input.rs 2071 71.81%
apps/client/src/lib.rs 1684 79.07%
apps/signalboxd/src/review_orchestration_runtime.rs 1333 2.56%
crates/persistence/src/model_execution.rs 1243 84.56%
crates/domain/src/turn_eligibility.rs 1223 90.33%
crates/persistence/src/runner_protocol.rs 1157 82.16%
apps/signalboxd/src/main.rs 1112 51.23%
crates/persistence/src/review_workflow.rs 997 77.35%
crates/tools-code-host/src/code_host/github.rs 992 76.49%
crates/tools-exec/src/bin/signalbox-exec-supervisor.rs 885 45.61%
crates/persistence/src/tool_loop.rs 732 76.49%
crates/tools-github/src/lib.rs 717 76.47%
crates/persistence/src/process_read.rs 693 81.92%
apps/client/src/presentation.rs 680 80.55%
apps/signalboxd/src/convergence_sweep_runtime.rs 649 16.47%
crates/domain/src/submit_input.rs 640 88.03%
apps/signalboxd/src/lib.rs 615 72.47%
crates/process-protocol/src/lib.rs 576 93.64%
apps/signalboxd/src/daemon_tools.rs 570 89.72%
crates/persistence/src/review_orchestration.rs 569 75.66%
crates/persistence/src/session_delegation.rs 543 77.63%
crates/application/src/model_execution.rs 541 85.81%

Measured at 61a6aa7e7e3786cccc574b61770f6e90b43e7396, the merge commit this pull request builds, whose head is 576d1dd0f817248757433b9904079ba011131237, by run 32606209437, which uploads the HTML report and LCOV as an artifact.

@codecov

codecov Bot commented Aug 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.59036% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.33%. Comparing base (9e70d0e) to head (576d1dd).

Files with missing lines Patch % Lines
crates/model-runtime/src/cli_redaction.rs 96.00% 1 Missing ⚠️
crates/model-runtime/src/redaction.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
Flag Coverage Δ
rust 82.92% <97.59%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants