You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Goal axis: 好用 / usability — permanently close the "advertised command does not exist" regression loop that both audits kept rediscovering.
The original failure (audit core-system-audit-2026-06-07.md, finding C0-2) was that .claude-plugin/plugin.json and .codex/hooks.json invoked a command that did not exist: the group is registered as @click.group("claude-code") (so hebb cc ... resolves to "No such command"), and there was never a write subcommand (UserPromptSubmit should call prompt). Meanwhile the programmatic installer wrote the correct strings, so two install paths silently drifted apart — breaking recall + memory for anyone who installed via the plugin marketplace or that shipped hooks.json. The newuser audit (newuser-experience-audit-2026-06-08.md, finding U3) found the same hebb claude-code write ghost command duplicated across the docs (cli.md, claude-code.md, README).
PR #24 fixed the strings, but a string fix does not prevent the next drift. This issue is about the durable guard.
Current state (verified against the tree)
A guard for the manifest + Dockerfile surface already landed in PR #24 (commit 9244265) — so this issue is narrowed to the remaining gap (docs / examples / READMEs), not the whole thing.
What already exists and passes (pytest tests/unit/test_audit_distribution.py → 3 passed):
tests/unit/test_audit_distribution.py:97test_plugin_hook_commands_resolve — extracts every hook command from .claude-plugin/plugin.json and resolves each against the live Click tree.
tests/unit/test_audit_distribution.py:107test_codex_hook_commands_resolve — same for .codex/hooks.json.
tests/unit/test_audit_distribution.py:117test_dockerfile_cmd_references_real_serve_command — same for the docker/DockerfileCMD chain.
The remaining gap — no test resolves hebb ... strings embedded in human-facing docs, which is exactly where the write ghost command lived (U3):
No test under tests/ references repo_pages, examples, or README (grep is empty).
repo_pages/**/*.md contains ~109 hebb <subcommand> strings (e.g. repo_pages/guide/claude-code.md:25,42,43,49), with an EN tree and a repo_pages/zh/ mirror.
examples/03_mcp_quickstart.md and examples/README.md contain hebb ... command strings (e.g. examples/03_mcp_quickstart.md:22,25,33).
README.md and README_ZH.md both carry hebb ... invocations.
So a future doc typo (hebb claude-code wrtie, a renamed/removed subcommand, etc.) would ship green today.
Proposed approach
Extend the existing tests/unit/test_audit_distribution.py (reuse its _resolve_command walker and from hebb.cli.main import main introspection — do not duplicate the tree logic) with a docs/examples/README coverage test:
Walk README.md, README_ZH.md, examples/**/*.md, and repo_pages/**/*.md (skip repo_pages/node_modules/).
Extract every hebb ... invocation from both fenced code blocks (```bash/```sh/plain) and inline `hebb ...` spans.
Normalize each invocation to a token path (strip the hebb executable; stop the sub-command path at the first -/-- option or shell metachar, matching the existing _hebb_invocations helper).
Assert each resolves to a registered command + subcommand via _resolve_command. Allow a small, explicit allowlist for intentionally illustrative/placeholder strings (e.g. hebb ... literally, or hebb <command> help syntax) so the test stays low-noise.
The test runs in the existing CI test job automatically (it lives under tests/unit).
Keep the manifest/Dockerfile tests as-is; this only adds the docs/examples/README dimension.
Acceptance criteria
A test introspects the live Click command tree from the hebb entrypoint (reuses the existing _resolve_command / main import, no duplicate tree logic).
It extracts and resolves hebb ... invocations from README.md, README_ZH.md, examples/**/*.md, and repo_pages/**/*.md (EN + repo_pages/zh/ mirror), covering both fenced blocks and inline code spans.
Introducing a fabricated bad command (e.g. hebb claude-code wrtie or hebb foo bar) in any covered doc fails the test; the current tree passes.
The manifest coverage (.claude-plugin/plugin.json, .codex/hooks.json) and Dockerfile coverage remain green (no regression to the existing three tests).
The new test runs in CI on every PR (verified via .github/workflows/ci.ymltest job picking up tests/unit).
Any intentional placeholder strings are handled via a narrow, documented allowlist rather than broad regex loosening.
Scope / out of scope
In scope: a CI-enforced assertion that every hebb ... string advertised in shipped manifests and human-facing docs/examples/READMEs resolves to a real Click command+subcommand.
Out of scope: validating option flags/argument values (e.g. that --scope user is a legal choice) — only the command-path is resolved, matching the existing _resolve_command semantics. Out of scope: localized prose translation correctness, and non-hebb shell commands in fences.
This is the highest-leverage usability fix in the backlog because it permanently closes the doc/command-drift regression loop that the core-system-audit-2026-06-07 (C0-2) and newuser-experience-audit-2026-06-08 (U3) audits independently rediscovered.
References
reports/audit/core-system-audit-2026-06-07.md (finding C0-2 — hebb cc write / wrong group name)
Context
Goal axis: 好用 / usability — permanently close the "advertised command does not exist" regression loop that both audits kept rediscovering.
The original failure (audit
core-system-audit-2026-06-07.md, finding C0-2) was that.claude-plugin/plugin.jsonand.codex/hooks.jsoninvoked a command that did not exist: the group is registered as@click.group("claude-code")(sohebb cc ...resolves to "No such command"), and there was never awritesubcommand (UserPromptSubmit should callprompt). Meanwhile the programmatic installer wrote the correct strings, so two install paths silently drifted apart — breaking recall + memory for anyone who installed via the plugin marketplace or that shippedhooks.json. The newuser audit (newuser-experience-audit-2026-06-08.md, finding U3) found the samehebb claude-code writeghost command duplicated across the docs (cli.md,claude-code.md, README).PR #24 fixed the strings, but a string fix does not prevent the next drift. This issue is about the durable guard.
Current state (verified against the tree)
A guard for the manifest + Dockerfile surface already landed in PR #24 (commit
9244265) — so this issue is narrowed to the remaining gap (docs / examples / READMEs), not the whole thing.What already exists and passes (
pytest tests/unit/test_audit_distribution.py→ 3 passed):tests/unit/test_audit_distribution.py:97test_plugin_hook_commands_resolve— extracts every hookcommandfrom.claude-plugin/plugin.jsonand resolves each against the live Click tree.tests/unit/test_audit_distribution.py:107test_codex_hook_commands_resolve— same for.codex/hooks.json.tests/unit/test_audit_distribution.py:117test_dockerfile_cmd_references_real_serve_command— same for thedocker/DockerfileCMDchain.tests/unit/test_audit_distribution.py:26_resolve_command(...)— the shared Click-tree walker (from hebb.cli.main import main); reused below..github/workflows/ci.yml:51-55runspytest tests/unit ...on every OS/Python matrix cell.Grounding of the shipped strings (all correct today, confirming PR #24 landed):
.claude-plugin/plugin.json:20,32,44→hebb claude-code recall/prompt/stop..codex/hooks.json:9,18,27→hebb claude-code recall/prompt/stop.src/hebb/integrations/claude_code/cli.py:8→@click.group("claude-code"); subcommands at linesinstall(20),uninstall(34),recall(42),prompt(50),stop(58). Live introspection confirmsclaude-codesubs =install, prompt, recall, stop, uninstall.The remaining gap — no test resolves
hebb ...strings embedded in human-facing docs, which is exactly where thewriteghost command lived (U3):tests/referencesrepo_pages,examples, orREADME(grep is empty).repo_pages/**/*.mdcontains ~109hebb <subcommand>strings (e.g.repo_pages/guide/claude-code.md:25,42,43,49), with an EN tree and arepo_pages/zh/mirror.examples/03_mcp_quickstart.mdandexamples/README.mdcontainhebb ...command strings (e.g.examples/03_mcp_quickstart.md:22,25,33).README.mdandREADME_ZH.mdboth carryhebb ...invocations.So a future doc typo (
hebb claude-code wrtie, a renamed/removed subcommand, etc.) would ship green today.Proposed approach
Extend the existing
tests/unit/test_audit_distribution.py(reuse its_resolve_commandwalker andfrom hebb.cli.main import mainintrospection — do not duplicate the tree logic) with a docs/examples/README coverage test:README.md,README_ZH.md,examples/**/*.md, andrepo_pages/**/*.md(skiprepo_pages/node_modules/).hebb ...invocation from both fenced code blocks (```bash/```sh/plain) and inline`hebb ...`spans.hebbexecutable; stop the sub-command path at the first-/--option or shell metachar, matching the existing_hebb_invocationshelper)._resolve_command. Allow a small, explicit allowlist for intentionally illustrative/placeholder strings (e.g.hebb ...literally, orhebb <command>help syntax) so the test stays low-noise.testjob automatically (it lives undertests/unit).Keep the manifest/Dockerfile tests as-is; this only adds the docs/examples/README dimension.
Acceptance criteria
hebbentrypoint (reuses the existing_resolve_command/mainimport, no duplicate tree logic).hebb ...invocations fromREADME.md,README_ZH.md,examples/**/*.md, andrepo_pages/**/*.md(EN +repo_pages/zh/mirror), covering both fenced blocks and inline code spans.hebb claude-code wrtieorhebb foo bar) in any covered doc fails the test; the current tree passes..claude-plugin/plugin.json,.codex/hooks.json) and Dockerfile coverage remain green (no regression to the existing three tests)..github/workflows/ci.ymltestjob picking uptests/unit).Scope / out of scope
hebb ...string advertised in shipped manifests and human-facing docs/examples/READMEs resolves to a real Click command+subcommand.--scope useris a legal choice) — only the command-path is resolved, matching the existing_resolve_commandsemantics. Out of scope: localized prose translation correctness, and non-hebbshell commands in fences.This is the highest-leverage usability fix in the backlog because it permanently closes the doc/command-drift regression loop that the
core-system-audit-2026-06-07(C0-2) andnewuser-experience-audit-2026-06-08(U3) audits independently rediscovered.References
reports/audit/core-system-audit-2026-06-07.md(finding C0-2 —hebb cc write/ wrong group name)reports/audit/newuser-experience-audit-2026-06-08.md(finding U3 —hebb claude-code writedoc drift)reports/design/capability-gap-roadmap-2026-06-11.mdtests/unit/test_audit_distribution.py(existing manifest/Dockerfile guard to extend)Filed from the capability-gap roadmap (reports/design/capability-gap-roadmap-2026-06-11.md).