Skip to content

Commit 077cd50

Browse files
GottZclaude
andcommitted
feat(blocktype): checkpoint-Evidence-Typ — Guard-feste Verankerung von ID-Referenzketten (M107)
Ursache (Befund 2026-07-20, ctx 019f7c7a): Compaction-Checkpoint-Manifeste und Transcript-Parts liefen auto-getypt als knowledge durch die Default- Guard-Lane (archive, 0.98) — aufeinanderfolgende Checkpoints derselben Session sind konstruktionsbedingt Fast-Duplikate. Der Guard archivierte Manifest + Part still; da jeder Read-Pfad NOT is_archived filtert, brachen ID-Referenzketten aktiver Compaction-Summaries (Dangling Manifest bei intaktem Datenbestand). Nebenwirkung: hex-dichte Part-Präfixe (~2 Byte/ Token) sprengten das 1024-Token-Slot-Fenster des Rerankers (exceed_context_size_error, 12 Vorkommen seit 2026-07-15). Fix: builtin-Typ checkpoint (M107, byte-äquivalent zu builtin.go, Golden- Drift-Gate) — retrieval=excluded, guard.check=false + guard.candidate=false, dream/digest/overview=false, classify per Titel-Präfix "compaction source" (prio 30); Writer sollen type=checkpoint weiterhin explizit setzen. Datenreparatur in M107: Retype der 125 Bestandsblöcke + Un-Archive der 64 guard-archivierten Evidence-Rows (neueste je (category,title,scope), NOT-EXISTS-Guard gegen den partiellen Unique-Index; Guard-Metadata bleibt erhalten). Re-Archive doppelt ausgeschlossen: guard_checked_at bleibt gesetzt UND der Typ verlässt die Guard-Allowlist. Gates: go test -short 32 Pakete grün; Integration blocktype/store/guard (testcontainers) grün; M107-Dry-Run gegen Live-DB in ROLLBACK-Transaktion (Seed 1 / Retype 125 / Un-Archive 64, Gegenprobe unverändert). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent a61c053 commit 077cd50

5 files changed

Lines changed: 149 additions & 16 deletions

File tree

docs/architecture.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@ Two further per-type dimensions are resolved in Go and passed to every call **ex
186186

187187
## Block-type registry (migration 072)
188188

189-
`context_block_types` is the declarative per-type behaviour registry of the workflow-engine line. The four M035 block-type classes (`knowledge`, `audit-trail`, `reference`, `system-meta`) ship as builtin seed rows (migration 072) whose JSONB configs reproduce today's hardcoded behaviour (retrieval damping, guard participation, dream linkability, digest/overview inclusion, classify rules); Welle I-C adds two more builtin rows (migration 084), the `issue`/`comment` workflow types (see below). The daemon starts on a compiled-in builtin set, overlays the DB rows at boot (merge — a deleted builtin row can never blank the default type) and hot-reloads on any table write via the same NOTIFY channel the settings use; edits via `psql` take effect without a restart.
189+
`context_block_types` is the declarative per-type behaviour registry of the workflow-engine line. The four M035 block-type classes (`knowledge`, `audit-trail`, `reference`, `system-meta`) ship as builtin seed rows (migration 072) whose JSONB configs reproduce today's hardcoded behaviour (retrieval damping, guard participation, dream linkability, digest/overview inclusion, classify rules); Welle I-C adds two more builtin rows (migration 084), the `issue`/`comment` workflow types, and migration 107 adds the `checkpoint` evidence type (see below). The daemon starts on a compiled-in builtin set, overlays the DB rows at boot (merge — a deleted builtin row can never blank the default type) and hot-reloads on any table write via the same NOTIFY channel the settings use; edits via `psql` take effect without a restart.
190190

191191
A boot that finds the table but cannot load it (corrupt row) degrades **loudly**: `/health` carries `blocktype_registry: "builtin-fallback"`, the overall status drops to `degraded`, and a 30s retry loop self-heals once the row is fixed.
192192

@@ -204,6 +204,8 @@ A boot that finds the table but cannot load it (corrupt row) degrades **loudly**
204204

205205
**Issue/comment workflow types (Welle I-C, migration 084).** `issue` is `full-pass` retrieval; its guard participates in **flag** mode (a duplicate issue is surfaced via a `possible_duplicate` flag, never auto-archived — §4.7) restricted to `same-scope` candidates, with per-type thresholds 0.97/0.90; dream links issues, but `digest.include=false` **and** `overview.include=false` keep a 10k+-issues/repo out of the topic-map and the Louvain overview clustering. `issue` carries a `workflow` state machine (`backlog`→`in-progress`→`done`, forge `open`/`closed` mapping) and a `structural_link_classes` allowlist (`references`, `duplicate-of`) — the new top-level policy vocabulary field (deliberately **not** `link_classes`, which is the distinct `dream.link_classes` semantic axis) that gates writes into `context_structural_links`; the five dream relationship names (`topical`, `factual`, `causal`, `recurrent`, `supersedes`) are **reserved** and reject at validation (graph-structural GA6 — a colliding class would make the `link_class` filter ambiguous and let a structural `supersedes` traverse while dream-supersedes is display-only); migration 105 preflights the existing corpus for colliding rows before a deploy can poison a registry reload, and a colliding row planted past it degrades that tenant loudly onto base (GA7); it is a declarative write allowlist consumed by the link-create path AND (since migration 103) by the daily-synthesis pipeline: `audit-trail` declares `references`, and `writeReportSourceLinks` persists report→source edges (origin `system`) for the blocks the Tagesbericht enumerates — fail-closed, an operator removing the class switches the edges off. `comment` is kept out of every autonomous pipeline (guard/dream/digest/overview all off). **Welle I-E flipped `comment` to the §4.1 target** (migration 085, in lockstep with `builtin.go`): `retrieval=aggregate-to-parent` (a ranked comment folds onto its parent issue — see [Aggregate-to-parent fold](#aggregate-to-parent-fold-wf-t11) above) + `parent.mode=required`/`comment-of`. Both mechanisms the target needs are live: the T11 fold consumer and the `parent_id` write path (`store.InsertCommentBlock`/`PutBlockParent`, I-D). Migration 084 stays the interim seed (`ON CONFLICT DO NOTHING`); 085 is the correcting `UPDATE` of the builtin `_global` row, and the registry drift gate diffs the decoded DB row against the compiled-in builtin so the two never separate. The interim shape (`retrieval=excluded` + `parent.mode=none`) that 084 planted while the mechanisms were unbuilt is retired.
206206

207+
**Checkpoint evidence type (migration 107).** `checkpoint` structurally anchors ID-referenced evidence chains — compaction-checkpoint manifests plus their transcript source parts, written by external agent plugins before a lossy context compaction. Resolution runs **exclusively over exact block IDs** (the manifest carries `source_block_ids` and a `parent_manifest_id` chain), so the type is out of **every** autonomous pipeline: `retrieval=excluded` (token-dense transcript parts would flood retrieval candidate sets — hex-heavy prefixes tokenize at ~2 bytes/token and overflow reranker slot windows), `guard.check=false` **and** `guard.candidate=false`, dream/digest/overview all off. The guard exemption is the load-bearing part: consecutive checkpoints of one session are near-duplicates **by construction** (manifest = boilerplate + IDs, parts overlap in the transcript window), and the default knowledge lane (archive mode, 0.98) auto-archived manifests/parts — with every read path filtering `NOT is_archived`, active summaries were left pointing at dangling IDs while the data sat intact (2026-07-20 incident). Blocks classify via the stable writer title prefix (`compaction source`, priority 30); writers SHOULD still assert `type=checkpoint` explicitly on store (`type_source='manual'`). Migration 107 also repairs the existing corpus: retype plus un-archive of the guard-archived evidence rows (newest per `(category,title,scope)`, guarded against the partial unique index; guard metadata stays for auditability — re-archive is impossible on both axes, `guard_checked_at` stays set and the type left the guard allowlist).
208+
207209
Since T10 the registry is **API-exposed** (no schema change) via the `type-*` manage family: `type-list`/`type-get` are open reads (scoped to `_global` ∪ the caller's own tenant namespace), `type-create`/`type-update`/`type-delete` are server-admin (tier 1: only the shipped `_global` namespace is writable). Configs are validated by the SAME decoder the reload path uses (422 with the offending key path); builtin rows are undeletable (their config IS editable), a delete is refused while ANY block references the type, and every mutation runs in one attributed transaction (`via='api'` + acting key) followed by a synchronous reload. Since T9 (migration 075) the legacy `is_meta` column + index dropped — the third type axis consolidated into the registry (`metadata.is_meta` survives as a JSONB classify-input key only).
208210

209211
**Tenant overlays (Stufe 2, WF T12 — no migration; the schema carries `UNIQUE(name, scope)` since M072).** On top of the `_global` base generation the registry keeps a lazy per-tenant cache — the exact `config.Store` overlay mechanism (immutable snapshot behind an atomic pointer, a monotone `baseGen` stamp per cached generation, single-flighted MISS builds, O(1) cache wipe on a base reload). `SnapshotForTenant(scope)` / `SnapshotForRequest(ctx)` resolve `_global ∪ tenant`, where a **tenant row WINS** on a name collision (**D6: "Overlay gewinnt" v1** — no monotonic narrowing; the alternative that would forbid a tenant lifting a `_global`-excluded type is a named open decision, pinned by a semantic-fixier test so a later flip is a visible break). A tenant with no own rows inherits the base pointer byte-for-byte, so the single-tenant / default-tenant server is functionally identical (zero per-tenant generation footprint). The request path derives the tenant scope from the auth context (never a caller argument, §5.4); a tenant-scope registry write drops **only** that tenant's cached generation via the NOTIFY dispatch (`InvalidateTenant`), while a `_global` write bumps `baseGen` and wipes the whole tenant cache. **Consumer reach:** request handlers (RRF/graph) resolve per caller tenant; the background **dream** and **digest** loops resolve per iterated tenant — the dream `PickBlock` is bound to the iterating tenant's scopes (`scope = ANY(owned)`) so tenant A's `dream.linkable`/allowlist never governs tenant B's blocks. The type policy is a **ranking/visibility** steering, **not** a confidentiality boundary (that stays scope + grants + sensitivity): a grantee's overlay steers the ranking of granted FOREIGN blocks too (Reader-Overlay wins). **Known Stufe-2 boundaries (documented, no leak):** the guard batch and the overview rebuild stay single global runs on the base allowlist (guard-state counts are global telemetry; overview visibility overrides need a per-tenant overview) — reads on both remain scope-filtered.

go/internal/blocktype/builtin.go

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,29 @@ func builtinPolicies() []Policy {
146146
Parent: ParentPolicy{Mode: ParentModeRequired, Relationship: "comment-of"},
147147
Classify: ClassifyRules{Priority: DefaultClassifyPriority},
148148
},
149+
// checkpoint: ID-anchored evidence blocks (compaction-checkpoint
150+
// manifests + transcript source parts, migration 107). Resolution runs
151+
// EXCLUSIVELY over exact block IDs (manifest content/metadata carry the
152+
// source_block_ids + parent_manifest chain), so the type stays out of
153+
// every autonomous pipeline: retrieval=excluded (transcript parts are
154+
// token-dense near-duplicates — in retrieval they flood candidate sets
155+
// and overflow the reranker slot window), guard.check=false AND
156+
// guard.candidate=false (consecutive checkpoints of one session are
157+
// near-duplicates BY CONSTRUCTION — the default archive lane silently
158+
// broke ID chains, the 2026-07-20 dangling-manifest incident),
159+
// dream/digest/overview all false. Classified by the stable writer
160+
// title prefix ("Compaction source …", priority 30 after system-meta/
161+
// audit-trail); writers SHOULD still set type=checkpoint explicitly.
162+
{
163+
Name: "checkpoint", Scope: globalScope, Builtin: true,
164+
Retrieval: RetrievalPolicy{Kind: RetrievalExcluded},
165+
Guard: GuardPolicy{Check: false, Candidate: false, Mode: GuardModeArchive, Candidates: GuardCandidatesAll},
166+
Dream: DreamPolicy{Linkable: false},
167+
Digest: DigestPolicy{Include: false},
168+
Overview: OverviewPolicy{Include: false},
169+
Parent: ParentPolicy{Mode: ParentModeNone},
170+
Classify: ClassifyRules{Priority: 30, TitlePatterns: []string{"compaction source"}},
171+
},
149172
}
150173
}
151174

go/internal/blocktype/registry_integration_test.go

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -166,10 +166,10 @@ func TestRegistryGolden_Integration(t *testing.T) {
166166
if err := pool.QueryRow(ctx, `SELECT count(*) FROM context_block_types`).Scan(&n); err != nil {
167167
t.Fatalf("count: %v", err)
168168
}
169-
// 6 since Welle I-C: 072 seeds 4, 084 seeds issue+comment; re-running 072
170-
// (ON CONFLICT DO NOTHING) adds none.
171-
if n != 6 {
172-
t.Errorf("rows after double-run = %d, want 6", n)
169+
// 7 since M107: 072 seeds 4, 084 seeds issue+comment, 107 seeds
170+
// checkpoint; re-running 072 (ON CONFLICT DO NOTHING) adds none.
171+
if n != 7 {
172+
t.Errorf("rows after double-run = %d, want 7", n)
173173
}
174174
})
175175

@@ -181,10 +181,10 @@ func TestRegistryGolden_Integration(t *testing.T) {
181181
AND metadata->>'via' = 'sql'`).Scan(&n); err != nil {
182182
t.Fatalf("audit count: %v", err)
183183
}
184-
// 6 since Welle I-C: 4 seed inserts from 072 + issue/comment from 084,
185-
// all via=sql (migration path, no api_key_id).
186-
if n != 6 {
187-
t.Errorf("block_type insert audit rows = %d, want 6 (seed inserts, via=sql)", n)
184+
// 7 since M107: 4 seed inserts from 072 + issue/comment from 084 +
185+
// checkpoint from 107, all via=sql (migration path, no api_key_id).
186+
if n != 7 {
187+
t.Errorf("block_type insert audit rows = %d, want 7 (seed inserts, via=sql)", n)
188188
}
189189
})
190190

go/internal/blocktype/set_test.go

Lines changed: 16 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,9 @@ func builtinTestSet(t *testing.T) *Set {
1616

1717
func TestBuiltinSetShape(t *testing.T) {
1818
s := builtinTestSet(t)
19-
// Six builtins since Welle I-C: the four M035 enum classes + issue/comment.
20-
want := []string{"audit-trail", "comment", "issue", "knowledge", "reference", "system-meta"}
19+
// Seven builtins since M107: the four M035 enum classes + issue/comment
20+
// (Welle I-C) + checkpoint (ID-anchored evidence, out of every pipeline).
21+
want := []string{"audit-trail", "checkpoint", "comment", "issue", "knowledge", "reference", "system-meta"}
2122
if got := s.Names(); !reflect.DeepEqual(got, want) {
2223
t.Errorf("Names() = %v, want %v", got, want)
2324
}
@@ -26,16 +27,19 @@ func TestBuiltinSetShape(t *testing.T) {
2627
}
2728
// Retrieval-visible = full-pass|damped|aggregate. issue is full-pass; comment
2829
// is aggregate-to-parent (I-E flip) — hence VISIBLE (it ranks in RRF, then
29-
// folds onto its parent issue). Only system-meta stays excluded.
30+
// folds onto its parent issue). system-meta + checkpoint stay excluded
31+
// (checkpoint evidence resolves over exact IDs only, M107).
3032
if got := s.VisibleTypes(); !reflect.DeepEqual(got, []string{"audit-trail", "comment", "issue", "knowledge", "reference"}) {
31-
t.Errorf("VisibleTypes() = %v (only system-meta must be excluded; comment is aggregate-visible)", got)
33+
t.Errorf("VisibleTypes() = %v (system-meta + checkpoint must be excluded; comment is aggregate-visible)", got)
3234
}
33-
// guard.check: the 4 builtins + issue; comment is OUT (guard.check=false).
35+
// guard.check: the 4 builtins + issue; comment + checkpoint are OUT
36+
// (guard.check=false — consecutive checkpoints are near-duplicates by
37+
// construction, the default archive lane broke ID chains, M107).
3438
if got := s.GuardCheckTypes(); !reflect.DeepEqual(got, []string{"audit-trail", "issue", "knowledge", "reference", "system-meta"}) {
35-
t.Errorf("GuardCheckTypes() = %v, want 4 builtins + issue (comment out)", got)
39+
t.Errorf("GuardCheckTypes() = %v, want 4 builtins + issue (comment + checkpoint out)", got)
3640
}
3741
if got := s.GuardCandidateTypes(); !reflect.DeepEqual(got, []string{"audit-trail", "issue", "knowledge", "reference", "system-meta"}) {
38-
t.Errorf("GuardCandidateTypes() = %v, want 4 builtins + issue (comment out)", got)
42+
t.Errorf("GuardCandidateTypes() = %v, want 4 builtins + issue (comment + checkpoint out)", got)
3943
}
4044
// The four M035 classes keep the guard bestand — archive persist + cross-
4145
// scope candidates. Builtins are constructed directly (not via DecodePolicy),
@@ -160,6 +164,11 @@ func TestClassifyMirrorsDecisionTree(t *testing.T) {
160164
{"title pattern", "Welle 41 Ergebnisse", nil, "audit-trail", true},
161165
{"title pattern case-insensitive", "SELF-AUDIT protokoll", nil, "audit-trail", true},
162166
{"no match falls to default", "pgvector tuning notes", map[string]any{"source": "claude-code"}, "knowledge", false},
167+
// checkpoint (M107): the stable writer title prefix classifies both
168+
// manifest and part rows; is_meta (priority 10) still wins over it.
169+
{"checkpoint manifest title", "Compaction source 20260712_205012_837f2c 1816f6b3ce6fc7e8 5e6b1698beab7814 manifest", nil, "checkpoint", true},
170+
{"checkpoint part title", "Compaction source candidate-e2e-x 00ff 00ff part 001 of 002", nil, "checkpoint", true},
171+
{"is_meta beats checkpoint pattern", "Compaction source irrelevant", map[string]any{"is_meta": true}, "system-meta", true},
163172
}
164173
for _, tc := range cases {
165174
t.Run(tc.name, func(t *testing.T) {

0 commit comments

Comments
 (0)