nogg

.pre-commit-config.yaml

@@ -1,4 +1,7 @@
 ---
+# mainbrain/ is a vendored tree (WORDLIB + OKComputer swarm) with its own
+# tooling and test suite; host lint/format/type hooks must not touch it.
+exclude: ^mainbrain/
 repos:
   - repo: https://github.com/mpalmer/action-validator
     rev: 7fa61514fb806fc9afd664b909e83584aca84f78 # v0.7.0

mainbrain/.gitignore

@@ -0,0 +1,4 @@
+# Counter unanchored patterns from the repo root .gitignore that would
+# swallow real source in the vendored apps (api/lib, src/lib in the swarm).
+!**/lib/
+!**/lib/**

mainbrain/README.md

@@ -0,0 +1,118 @@
+# MAINBRAIN — Unified Best-of-Both Build
+
+Assembled 2026-07-17 from two uploaded packages:
+
+- `Babafile.zip` → `CONVERSATION_HANDOFF_MASTERZIP_v1` (conversation handoff: docs, baseline trees, the OKComputer swarm app)
+- `OMEGA_MEGAKERNEL_PACKAGE.zip` → the evolved WORDLIB tree (semantic core + docking protocol era)
+
+This directory is the merge of the strongest parts of both, with duplicates
+eliminated by provenance analysis rather than guesswork.
+
+## Layout
+
+```text
+mainbrain/
+  wordlib/            The OMEGA Megakernel WORDLIB tree (canonical, newest in hand).
+                      Offline-first AI workstation: launcher brain, ETHER AI Flask hub,
+                      12-agent swarm, 5-layer reasoning stack, atomic self-evolution
+                      with rollback, deployment stage engine, docking/spec protocol,
+                      CNS deploy proof gates, 23 test modules.
+  okcomputer-swarm/   OKComputer Swarm Integrated (from Babafile). React/TypeScript/
+                      tRPC/Drizzle shell: OpenAI Agents SDK planner-worker-validator-
+                      synthesizer, Claude as adversarial critic, Hugging Face
+                      Transformers.js local embeddings, Supabase control plane.
+  ether-runtime/      Pure-stdlib durable task runtime — offline reconstruction of
+                      the missing EtherAI Absorption v11 artifact (top-5 #4): SQLite
+                      WAL journal + transactional outbox, consumer-group stream with
+                      idle reclaim, deterministic jittered retries, allowlisted task
+                      types, at-least-once with terminal-state dedup. 24 tests.
+  handoff/            The conversation-handoff record: start-here docs, the three
+                      runtime/memory design conversations, MASTERZIP v11 domain spec,
+                      and integrity manifests of the original bundle.
+  TOP5.md             The top-5 conversations/models/code, presented in detail.
+```
+
+## De-duplication decisions (what was NOT copied, and why)
+
+1. **`WORDLIB_v29_BASELINE` (Babafile) — dropped.** A full recursive diff against
+   the OMEGA tree showed zero unique files: every baseline file exists in OMEGA,
+   and all 8 differing files are strictly newer/larger in OMEGA (e.g.
+   `deploy_check.py` 319 → 1,442 lines with the CNS proof-gate work). The only
+   baseline-unique content was `__pycache__` bytecode.
+2. **`90_ORIGINAL_UPLOADS` — dropped.** Retained in the original zip for rollback;
+   every member is superseded by a canonical working tree or a handoff doc here.
+3. **`40_DATABASE_AND_PROVIDER_STATE` — dropped.** All four files are
+   byte-identical to copies shipped inside `okcomputer-swarm/` (verified with
+   `cmp`). The operative copies live with the app.
+4. **Runtime state excluded by the tree's own `.gitignore`**: 245 self-editor
+   `.bak` snapshots, event/genome/memory JSON, logs. These are machine-specific
+   and regenerated; the self-healer recreates missing directories on boot.
+
+## How the two worlds fit (adapter-first federation)
+
+Per `handoff/00_START_HERE/MERGE_AND_EXECUTION_PLAN.md`: OKComputer is the
+connected orchestration shell; WORDLIB is the offline reasoning/memory/proof
+foundation. They federate through adapters and evidence gates — not by pasting
+one tree into the other.
+
+```text
+User / UI
+  -> okcomputer-swarm (connected shell)
+       -> OpenAI Agents SDK: planner, workers, validator, synthesizer
+       -> Claude: adversarial critic (server-side key only)
+       -> Transformers.js: local semantic routing (mxbai-embed-xsmall-v1)
+       -> Supabase: control.swarm_runs / swarm_events / swarm_checkpoints
+  -> wordlib (offline brain)
+       -> ETHER AI hub (Flask, :5757) + RAG
+       -> 12-agent swarm, reasoning guards, uncertainty quantifier
+       -> EvolutionManager: atomic multi-file self-edit with verified rollback
+       -> docking/: spec protocol, patch payloads, CNS deploy proof gates
+```
+
+Swarm state machine (both worlds observe it):
+`INTAKE -> PLAN -> ROUTE -> EXECUTE -> CRITIQUE -> VALIDATE -> PERSIST -> COMPLETE | FAILED`
+
+## Quickstarts
+
+**wordlib** (offline brain; Windows-first, Linux parity):
+
+```text
+START_HERE.bat        one-time polished auto-install (8 stages)
+RUN_ME.bat            daily driver menu
+python tests/run_tests.py   test suite without pytest
+```
+
+**okcomputer-swarm** (connected shell; needs Node + server-side keys):
+
+```bash
+cd okcomputer-swarm
+cp .env.example .env   # keys go in the server environment only, never committed
+npm install && npm run verify:integration && npm run dev
+```
+
+## Safety rules carried over (non-negotiable)
+
+- Never expose provider keys to browser code, logs, prompts, or this repo.
+- Never merge WORDLIB self-editing into active execution without the gate order:
+  `SPEC-CRYPTOGRAPHIC_BINDING -> SPEC-DEPLOY_PROOF_GATE ->
+  SPEC-TESTED_PATCH_EXECUTION_SANDBOX -> human approval`.
+- MASTERZIP v11 stays **VAL-0 / specification state**: audit it into implemented
+  code / placeholders / unsupported claims / evidence-required before any
+  physical or commercial claim.
+- GremlinAgent stays propose-only (Theater Mode); GuardianAgent veto stands.
+
+## The frontier beyond this snapshot
+
+Connector sweep on 2026-07-17 found the ecosystem has release *receipts* newer
+than any code in hand — the artifacts themselves stayed in chat handoffs:
+
+- **v30 Mainbrain** release receipt (Drive) — 103/103 tests, Codex Masterfile hashes.
+- **v31 MARM + Filesystem MCP** (Notion) — 118/118 tests, sidecar architecture.
+- **v32 Memory Integrity** (Drive doc + Linear project `WORDLIB Memory Integrity
+  v32`) — 132/132 tests, Wolfram-verified retry/stability kernel, Black Vault
+  repair-first plan. Linear: MOH-17 done, MOH-16 in progress, MOH-18 todo,
+  MOH-19 blocked.
+
+This `mainbrain/` tree is therefore the newest **executable** consolidation;
+the v30–v32 receipts document work whose code should be recovered from those
+chat handoffs and layered on top, gate by gate.

mainbrain/TOP5.md

@@ -0,0 +1,167 @@
+# The Top 5 Conversations, Models, and Code
+
+Ranked by executable weight and how much of the ecosystem depends on them.
+Sources: the two uploaded packages, plus a live connector sweep (Google Drive,
+Notion, Linear, Gmail) on 2026-07-17.
+
+---
+
+## 1. WORDLIB → OMEGA Megakernel — the offline brain
+
+**Code (in hand):** `mainbrain/wordlib/` — the single largest, most evolved asset.
+
+**Conversation thread:** `wordlib/CLAUDE_BRIEFING.md` — a 1,200-line evolution
+log, v1 through v29, written as a running handoff between Claude sessions.
+It is the highest-signal conversation in either package.
+
+What the code actually contains, all verified-real per its own honesty log:
+
+- `launcher.py` — the one brain: venv, services, hub, menu, one-hit install.
+- `src/ether_core.py` — EtherCore blob brain; `EvolutionManager.evolve_files()`
+  gives **atomic multi-file self-modification** with validate → snapshot →
+  deploy → test → verified rollback (a real rollback bug was caught by the
+  test suite and fixed at v13.8).
+- `src/agents/` — ClawOrchestrator + 12 agents (Architect, Code, Test, Deploy,
+  Guardian veto, Researcher, Reasoning, Math, Absorption, Gremlin propose-only,
+  Market, EvolutionArchitect).
+- `src/reasoning/` — 5-layer honest reasoning stack: output-quality guard,
+  sympy math validator, ast code reasoner, structured-output repair+cache,
+  Wilson/Brier/ECE uncertainty quantifier, plus quantum-*inspired* (classical,
+  honestly labeled) optimizer and an Ollama LLM-as-judge.
+- `src/deployment/` — stage engine: idempotent, resumable, retry w/ backoff,
+  rollback, self-repair; absorption scanner + environment intelligence.
+- `docking/` + `core/contracts/` + `deploy_check.py` (1,442 lines) — the
+  spec/docking protocol with CNS deploy proof gates (see #3).
+- `tests/` — 23 modules, pytest-free runner for USB deployment.
+- Three WCAG 2.2 AA UIs: `/panel`, `/console` (reasoning-transparent),
+  `/nephilim` (living orbital UI) — all guarded by tests that fail on any
+  phantom endpoint.
+
+**Models (local, confirmed tags):** `dolphin-2.9-mistral-7b` Q4_K_M (primary),
+`dolphin3:8b-llama3.1-q4_K_M` (creative), `qwen2.5-coder:7b-instruct-q4_K_M`
+(code), `Meta-Llama-3-8B-Instruct` Q4_K_M (classroom-safe),
+`nomic-embed-text` (RAG embeddings).
+
+---
+
+## 2. OKComputer Swarm Integrated — the connected orchestration shell
+
+**Code (in hand):** `mainbrain/okcomputer-swarm/` — React 18 + TypeScript +
+tRPC + Drizzle + Vite, with the swarm runtime under `api/ai/`.
+
+**Conversation thread:** `handoff/00_START_HERE/HIGH_SIGNAL_CONVERSATION_RECORD.md`
+— the consolidation conversation that fused MASTERZIP domain knowledge with
+the swarm shell and made five architectural decisions:
+
+1. **OpenAI Agents SDK is primary** — planner, specialist workers, validator,
+   synthesizer (`api/ai/openai-swarm.ts`, `execute-swarm.ts`).
+2. **Claude is the adversarial critic**, never primary authority
+   (`api/ai/claude-critic.ts`), output routed back into validation.
+3. **Hugging Face is local-first** — Transformers.js semantic router with
+   `mixedbread-ai/mxbai-embed-xsmall-v1`, remote loading disabled
+   (`api/ai/hf-local-router.ts`).
+4. **Supabase is the control plane, not bulk storage** —
+   `control.swarm_runs` / `swarm_events` / `swarm_checkpoints` behind RLS with
+   service-role-only RPCs (`SUPABASE_SWARM_CONTROL_MIGRATION.sql`, applied and
+   smoke-tested live during the conversation).
+5. **Canva is a documentation surface only** (4 operator-guide candidates).
+
+The state machine (`api/ai/state-machine.ts` + its test):
+`INTAKE → PLAN → ROUTE → EXECUTE → CRITIQUE → VALIDATE → PERSIST → COMPLETE | FAILED`.
+Security corrections from the conversation: simulated provider outputs removed
+as an execution path, credentials server-side only, auth + ownership required
+on all run reads/writes.
+
+---
+
+## 3. Semantic Core + Docking Protocol — spec-driven self-improvement with proof gates
+
+**Conversation thread:** `handoff/30_RUNTIME_AND_MEMORY_DESIGNS/WORDLIB semantic
+core update.txt` — the fork-B decision ordering the gate chain:
+`SPEC-CRYPTOGRAPHIC_BINDING` (done) → `SPEC-DEPLOY_PROOF_GATE` (ordered) →
+`SPEC-TESTED_PATCH_EXECUTION_SANDBOX` (next wound to close).
+
+**Code (in hand, inside `mainbrain/wordlib/`):** this conversation was
+*implemented* in the OMEGA tree — the clearest proof OMEGA supersedes the
+Babafile baseline:
+
+- `docking/` — spec registry, lifecycle, validator, auditor, patch applier,
+  failure memory, side letters, improvement pipeline.
+- `core/contracts/docking/` — typed contracts: SpecDocument, PatchPayload,
+  ImprovementProposal/Record, FailureRecord, ChangeTrace, SideLetter.
+- `deploy_check.py` grew 319 → 1,442 lines: deploy-time CNS proof-gate checks
+  (`canonical_patch_digest()` determinism, digest-mismatch rejection, no
+  whitelist/approval bypass, `--json` mode, non-zero exit on gate failure).
+- Tests: `test_docking_protocol.py`, `test_patch_payload.py`,
+  `test_sideletter_protocol.py`, `test_closed_loop_improvement.py`, etc.
+
+**Models thread:** `Consider best setup.txt` — the upgrade recommendation:
+one ~32B-class reasoning brain (Qwen3.5-32B quantized) + a 7–9B fast router,
+and a hybrid **LanceDB (vectors) + FalkorDB/Neo4j (graph/ontology)** semantic
+core instead of a plain vector DB.
+
+---
+
+## 4. EtherAI Absorption System v11 — the durable task runtime (design in hand, artifact missing)
+
+**Conversation thread:** `handoff/30_RUNTIME_AND_MEMORY_DESIGNS/EtherAI
+Absorption System.txt` — a complete production design record:
+
+- Redis Streams consumer-group dispatch, `XREADGROUP` + `XAUTOCLAIM` crash
+  recovery, SQLite WAL transactional task **outbox**, worker leases,
+  dead-letter records, health-gated Compose startup.
+- Honest **at-least-once** delivery claim (not exactly-once), duplicate window
+  controlled by stable task_id + leases + terminal-state dedup.
+- Wolfram-verified retry policy: delays 1/2/4/8/16s, deterministic ±20% jitter
+  from `SHA-256(task_id + attempt)`, reclaim threshold 60s.
+- Security correction: the proposed `/execute_verified` arbitrary-shell
+  endpoint was **removed**; only `absorb_text` and `verify_artifact` remain.
+
+**Status:** the v11 starter zip/wheel referenced by the record is **not present
+in any package** (`handoff/00_START_HERE/MISSING_ARTIFACTS.md`).
+
+**Rebuilt:** `mainbrain/ether-runtime/` now implements the v11 durability model
+from this record — pure stdlib, fully offline: SQLite WAL journal +
+transactional outbox, consumer-group stream with 60s idle reclaim, execution
+leases, the exact 1/2/4/8/16s ±20% deterministic-jitter retry ladder,
+dead-lettering, and only the two validated task types. 24 tests prove the
+outbox atomicity, the at-least-once duplicate window, lease exclusion, and the
+retry math. Redis/HTTP layers remain honest future adapters — see its README.
+
+---
+
+## 5. MASTERZIP v11 / ALUM-BOO — the engineering domain spec (VAL-0)
+
+**Document (in hand):** `handoff/20_DOMAIN_SPECIFICATIONS/` — the combined
+engineering handover PDF + extracted text: domain rules, blueprints, FMEA,
+validation ladders, T1–T28 tool concepts, prototype scripts.
+
+**Standing order from the conversation record:** treat as **VAL-0 /
+specification state**. The first real swarm request is the audit:
+
+> Audit MASTERZIP v11. Separate implemented code, placeholders, unsupported
+> claims, and the exact evidence required before VAL-1.
+
+Physical validation ladder VAL-0 → VAL-9 (geometry → materials → control sim →
+mock-up → powered tests → RF → operational cycles). The swarm may generate
+analyses and test plans; it may not certify a build or sale.
+
+**Connector context:** Drive also holds `ALUMBOO_R17_MASTERFILE.md` (2026-07-15)
+— the strategy pivot to the aluminum-glass cash-core (AGFS) with Class-A/Class-B
+feedstock separation. Same doctrine: no structural claims without tests.
+
+---
+
+## Beyond the zips: the live frontier (connector sweep, 2026-07-17)
+
+The newest *receipts* found in connectors — code for these stayed in chat
+handoffs and should be recovered next:
+
+| Version | Where found | Evidence |
+| --- | --- | --- |
+| v30 Mainbrain release | Drive doc + Notion pages | 103/103 tests, artifact SHA-256 list, deployer SKILL.md |
+| v31 MARM + Filesystem MCP | Notion cockpit + evidence pages | 118/118 tests, dual-STDIO sidecar architecture |
+| v32 Memory Integrity | Drive doc + Linear project | 132/132 tests, Wolfram stability kernel, Black Vault repair-first plan; MOH-17 ✅, MOH-16 🔄, MOH-18 ⏳, MOH-19 ⛔ |
+
+The `mainbrain/` tree in this repo is the newest **executable** consolidation
+of everything above.

mainbrain/ether-runtime/README.md

@@ -0,0 +1,59 @@
+# ether-runtime — durable task runtime (pure stdlib, offline)
+
+Reconstruction of the missing **EtherAI Absorption System v11** artifact —
+top-5 item #4. The original v11 starter zip/wheel referenced by the design
+record (`../handoff/30_RUNTIME_AND_MEMORY_DESIGNS/EtherAI Absorption
+System.txt`) was never present in any package; this implements its durability
+model from that record, with zero third-party dependencies, so it runs and
+tests fully offline.
+
+## What it implements from the v11 record
+
+| v11 design element | Here |
+| --- | --- |
+| SQLite WAL task journaling | `store.py` — WAL journal, terminal status authoritative |
+| Transactional task outbox | submit = task + outbox row in ONE transaction |
+| Outbox claim leases | publishers claim rows under a lease; no double publish |
+| Consumer groups (`XREADGROUP`) | `read_group()` — pending-entries list per group |
+| Crash recovery (`XAUTOCLAIM`) | `autoclaim()` — transfers entries idle ≥ 60s |
+| Worker execution leases | one runner per task; expiry frees crashed work |
+| Retry ladder 1/2/4/8/16s | `retry.py`, nominal total 31s < 60s reclaim |
+| Deterministic ±20% jitter | derived from `SHA-256(task_id + attempt)` |
+| Dead-letter records | `dead_lettered` status after max attempts |
+| At-least-once, not exactly-once | duplicate window controlled by stable task ids, task uniqueness, leases, terminal-state dedup (tested) |
+| Only validated task types | `absorb_text`, `verify_artifact`; unknown kinds rejected at submit — no arbitrary-execution endpoint, same boundary the v11 review enforced |
+
+## Usage
+
+```bash
+python -m ether_runtime submit-absorb --source NOTE --text "durable text"
+python -m ether_runtime submit-verify --file rel/path.bin=<sha256>
+python -m ether_runtime work            # drain once
+python -m ether_runtime status          # journal counts + retry policy
+python -m ether_runtime doctor          # findings with exact fixes
+```
+
+Run the tests (stdlib + pytest only):
+
+```bash
+cd mainbrain/ether-runtime && python -m pytest -q
+```
+
+## Honest divergences from v11
+
+- **No Redis / no Docker / no FastAPI here.** The stream + pending-entries
+  tables are a single-node, in-database stand-in for Redis Streams with the
+  same claim/ack/reclaim semantics. A Redis adapter can implement the same
+  operations later without touching worker logic; an HTTP gateway is a thin
+  layer over `TaskRuntime.submit()` when a connected deployment needs one.
+- **Single-node coordination.** SQLite is not distributed consensus — same
+  limit the v11 record states for its own SQLite authority.
+- The Wolfram-verified numbers (31s nominal retry total vs 60s reclaim) are
+  re-derived at startup by `retry.validate_policy()`, which raises if the
+  invariant is ever broken by edit.
+
+## Integration path (merge plan Phase 3)
+
+Swarm planner → bounded task envelope (`submit`) → durable outbox → stream →
+worker lease → result/evidence record (`result` on the journal row) → swarm
+validator. Allowlisted task types only.

mainbrain/ether-runtime/ether_runtime/__init__.py

@@ -0,0 +1,37 @@
+"""ether_runtime -- pure-stdlib durable task runtime.
+
+Offline reconstruction of the missing EtherAI Absorption System v11 artifact
+(design record: mainbrain/handoff/30_RUNTIME_AND_MEMORY_DESIGNS/). SQLite WAL
+journal + transactional outbox + consumer-group stream with idle reclaim,
+deterministic jittered retries, allowlisted task types, at-least-once
+delivery with terminal-state dedup. See README.md for honest limits.
+"""
+
+from .retry import (
+    DEFAULT_MAX_ATTEMPTS,
+    RECLAIM_IDLE_SECONDS,
+    RETRY_DELAYS,
+    jitter_factor,
+    retry_delay,
+    validate_policy,
+)
+from .runtime import GROUP, TaskRuntime, Worker
+from .store import Task, TaskStore, stable_task_id
+from .tasks import ALLOWED_KINDS, PayloadError
+
+__all__ = [
+    "ALLOWED_KINDS",
+    "DEFAULT_MAX_ATTEMPTS",
+    "GROUP",
+    "PayloadError",
+    "RECLAIM_IDLE_SECONDS",
+    "RETRY_DELAYS",
+    "Task",
+    "TaskRuntime",
+    "TaskStore",
+    "Worker",
+    "jitter_factor",
+    "retry_delay",
+    "stable_task_id",
+    "validate_policy",
+]