feat(engineering): add a verification stage and a second provider to hivemind - #1003
feat(engineering): add a verification stage and a second provider to hivemind#1003Hanishchow wants to merge 4 commits into
Conversation
…hivemind Two weaknesses in the merged version, both found by running it. Every worker claim had to be re-checked by hand, which cost more than the delegation saved. oc-verify.mjs sends the claims to a verifier that re-derives them from the same files and tries to refute them, deliberately on a different model so its blind spots do not match the worker's. It returns counts plus a needs_review list, so only refuted and unsupported claims need a human. And opencode's free tier is not dependable: models 404 at the provider and return empty bodies. or-worker.mjs speaks the identical one-JSON-line contract over OpenRouter, so oc-verify, oc-status and oc-aggregate work unchanged. oc-worker.mjs also gains --json, because workers bury their JSON in fenced blocks after a paragraph of prose; it now extracts the value and gives one corrective retry instead of every caller writing the same regex.
…i#979 Overwriting oc-worker.mjs with my copy dropped the // auditor:ignore-line note added upstream after the merge, which would have reintroduced a CRITICAL CMD-INJECT finding in the security gate. Restored verbatim. oc-verify.mjs imports child_process for the same reason - it invokes oc-worker.mjs to run the verifier - so it carries its own justification rather than borrowing that one. Verified with the repo's own skill_security_auditor.py --strict: PASS, 0 critical.
Gate G10 failed because the new agent file had no `name` in its frontmatter. The agents already in this repo were given namespaced names after alirezarezvani#979 merged (hive-scout, hive-coder, hive-tester), so this one is hive-verifier. Verified with scripts/check_frontmatter.py: 0 errors, same single warning the existing agent files produce.
Gate G3 derives the agent count from the tree and compares it against the headline numbers in CLAUDE.md and the README badge. Adding one agent file makes those read 118 where the tree now has 119. CONTRIBUTING asks contributors not to touch index files, but G3 blocks any PR that adds an agent without this, so the two hunks are here to make CI pass - revert them and bump it yourself if you would rather.
|
CI status and one thing I need your call on. Fixed after opening:
Still red: G3 (counter derivation), and I don't think I should fix it. Adding one agent file takes the derived
But
CONTRIBUTING says not to modify Three ways I'm happy to go:
VirusTotal is failing the same way it did on #979 — fork PRs don't get the API secret. Tessl I have no visibility into. Happy to take whichever path is least work for you. |
Summary
engineering/hivemind, plus a--jsonfix in the worker script.The verification stage
Every worker claim had to be re-checked by hand, which cost more than the delegation saved.
scripts/oc-verify.mjssends the claims to a verifier that re-derives them from the same files and tries to refute them. It returns{confirmed, refuted, unsupported, unverified, needs_review[]}— you readneeds_reviewonly; confirmed claims come back as a count rather than prose.Four rules keep it from becoming a rubber stamp, and they're documented in SKILL.md:
unverified, not passed.The second provider
opencode's free tier is not dependable — models return provider 404s and empty bodies.
scripts/or-worker.mjsspeaks the identical one-JSON-line contract over OpenRouter, sooc-verify,oc-statusandoc-aggregatework against it unchanged. Key comes fromOPENROUTER_API_KEYor~/.claude/.openrouter_key, is never logged, and never appears in output.SKILL.md now includes a measured table of which models actually work: six free-model attempts failed for five different reasons, while the same free model answered a plain-text question correctly at zero cost. The rule that falls out is capability, not availability — structured output needs a paid model, prose does not.
The
--jsonfixWorkers bury their JSON in fenced blocks after a paragraph of prose.
oc-worker.mjs --jsonnow extracts the value and gives one corrective retry, instead of every caller writing the same regex. Unit-tested against fenced, prose-prefixed, bare-array, bare-object and fence-without-language shapes.Checklist
devname+descriptiononlyassets/commands/layout untouched — additive only, nothing removednode --checkScripts remain
.mjsrather than stdlib Python, as in #979. Happy to adjust if that's now a problem.