Skip to content

docs: make the prose match the shipped behaviour, and pin the version copies - #375

Merged
Faisal-Fayaz merged 1 commit into
mainfrom
docs/sync-with-shipped-behaviour
Oct 7, 2026
Merged

Faisal-Fayaz merged 1 commit into
mainfrom
docs/sync-with-shipped-behaviour

Conversation

@Faisal-Fayaz

Copy link
Copy Markdown
Owner

Two of these would have led a user to write something that silently does nothing

docs/plugins.md was still a design document

It said "Status: design — no implementation here; #49 implements against this doc." The feature shipped long ago. Worse, its example manifest was wrong in a way that fails silently:

params:                          # ← a YAML mapping
  city: {type: string, required: true}

plugins.py only handles params when the value is a string, so a mapping leaves the parameter set empty, {city} is never substituted, and the tool still appears in the schema with no required arguments. Nothing warns.

Rewritten as shipped, showing the working inline form (params: city: string required) and adding a Deliberate differences table recording each divergence from the original design and why.

The documented shell fallback does not exist

"Anything outside the allowlist falls back to the shell tool, which keeps its approval gate + hard-refusal patterns."

There is no fallback and no escalation — tool_exec returns its own Blocked: … string and the command does not run. The code is right and the doc was wrong: implicit escalation to a more powerful tool is a worse default than refusing.

docs/mcp-client.md omitted the key that refuses your tools

Default trust is untrusted, a tool needs a literal readOnlyHint: true, and the refusal message tells you to set trust = "full" — a key that appeared nowhere in the file. inherit_env was absent entirely. Both are now documented, with the three gates (trust / readOnlyHint / approval) spelled out, including that an unrecognised trust value silently normalises to untrusted.

Two web_search claims were false

README and docs/egress.md both said the allowlist covers "every hit it follows". tool_web_search requests only html.duckduckgo.com and returns result URLs as text — it never fetches a hit.

SIDEKICK_TRUST_REPO was undocumented

A security gate: a repo's .sidekick/commands/*.md expands !`cmd` into sh -c with no approval prompt, and is silently ignored without it. A user had no way to discover why their repo's commands did nothing.

Version copies — all five had drifted

was now
AUR pkgver 0.26.0 + sdist hash for 0.26.0 0.29.0, URL + sha256 verified against the real artifact
AUR publish flow "tag vX.Y.Z" by hand the workflow tags; on.push filters on branches, so a tag push runs nothing
ROADMAP.md Next v0.28.0 (already shipped) v0.30.0, with 0.28/0.29 recorded
website roadmap 8 releases stale matches ROADMAP.md
CONTRIBUTING.md checklist demanded a test-count badge gate removed in #319 names the real required checks

Tests (+14)

Mechanical agreements only: version copies agree, documented keys exist, the params form shown is the form the parser accepts, the docs/examples/ manifests don't use the broken mapping form, and no removed gate is still referenced.

Deliberately not asserting that prose is true — a doc can be confidently wrong in a way no test can see, which is why the high-severity ones were verified against the code first.

sabotage caught
revert plugins.md to design-doc framing ✅
remove the trust key documentation ✅

1381 passed (+14), ruff clean, mypy clean, coverage 79.92%.

… copies

An audit of the docs found prose describing behaviour that does not exist. Two
of these would have led a user to write something that silently does nothing.

**docs/plugins.md was still a design document.** It said 'Status: design -- no
implementation here; #49 implements against this doc', and the feature has
shipped. Worse, its example manifest was wrong in a way that fails silently:

    params:                          # a YAML mapping
      city: {type: string, required: true}

The parser only handles params when the value is a string, so a mapping leaves
the parameter set empty, {city} is never substituted, and the tool still lists
in the schema with no required arguments. Nothing warns. Rewritten as shipped,
with the working inline form shown and a 'Deliberate differences' table
recording each divergence from the original design and why.

**The shell fallback does not exist.** The doc said a command outside
ALLOWED_BINARIES 'falls back to the shell tool, which keeps its approval gate +
hard-refusal patterns'. There is no fallback and no escalation -- tool_exec
returns its own 'Blocked: ...' string and the command does not run. Implicit
escalation to a more powerful tool is a worse default than refusing, so the code
is right and the doc was wrong.

**docs/mcp-client.md omitted the key that refuses your tools.** Default trust
is 'untrusted', a tool needs a literal readOnlyHint: true, and the refusal
tells you to set trust = "full" -- a key that appeared nowhere in the file.
inherit_env was absent entirely. Both documented, with the three gates
(trust / readOnlyHint / approval) spelled out, including that an unrecognised
trust value silently normalises to untrusted.

**Two web_search claims were false.** Both README and docs/egress.md said the
allowlist covers 'every hit it follows'. tool_web_search requests only
html.duckduckgo.com and returns result URLs as text; it never fetches a hit.
Following one is read_url, which is governed on its own.

**SIDEKICK_TRUST_REPO was undocumented** -- a security gate, since a repo's
.sidekick/commands/*.md expands !`cmd` into sh -c with no approval and is
silently ignored without it. A user had no way to find out why their repo's
commands did nothing.

**Version copies had all drifted.** AUR pkgver was 0.26.0 with a hardcoded
sdist hash for that version (now 0.29.0, URL and sha256 verified against the
real artifact) and documented a manual tag step the release workflow does
instead -- while on.push filters on branches, so a tag push runs nothing.
ROADMAP 'Next' pointed at v0.28.0, already shipped. The website roadmap was
eight releases stale. CONTRIBUTING's release checklist still demanded a
test-count badge gate that #319 removed, so a release PR was being judged
against gates that no longer exist.

14 new tests assert the mechanical agreements -- version copies agree,
documented keys exist, the params form shown is the form the parser accepts,
and no removed gate is still referenced. Deliberately not asserting that prose
is true: a doc can be confidently wrong in a way no test can see, which is why
the high-severity ones were verified against the code first.

Sabotages: reverting plugins.md to design-doc framing fails; removing the trust
key docs fails. 1382 passed (+14), ruff clean, mypy clean, coverage 79.92%.
@Faisal-Fayaz
Faisal-Fayaz merged commit 0867ccc into main Oct 7, 2026
12 checks passed
@Faisal-Fayaz
Faisal-Fayaz deleted the docs/sync-with-shipped-behaviour branch October 7, 2026 07:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant