UPDATE/DELETE on compressed partitions (DML P2 + tombstone DELETE)

[SferaDev]

UPDATE/DELETE on compressed partitions (DML P2 + tombstone DELETE)

Stacks on #34 (DML P1, feat/compressed-insert), which stacks on #33: #33 ← #34 ← this PR. Base branch is feat/compressed-insert, not main.

What

Compressed partitions accept plain UPDATE / DELETE / MERGE (including data-modifying CTE forms). Two mechanisms:

P2 — decompose-on-write (UPDATE, and DELETEs that don't qualify for tombstones). The ExecutorStart interceptor walks every ModifyTable node, locates candidate segments with the read path's pruning pipeline (dml_candidate_segments → load_segments_heap), and decomposes them back into ordinary heap rows (decompose_segments_for_dml in src/compress.rs) inside the user's transaction, before the planned DML executes. Everything is plain WAL-logged heap DML in one transaction — MVCC, rollback, and crash recovery need no extension-side machinery: a crash mid-decompose leaves the segment intact (the meta-row delete and the heap inserts roll back together). A whole-segment DELETE fast path drops segments without restoring rows. Its guards disable it whenever something must observe the deleted rows: RETURNING on that ModifyTable (checked per node, so CTE-level DELETE ... RETURNING decomposes too), row-level DELETE triggers, or REFERENCING OLD TABLE transition relations on the leaf or the named target; and every qual must be batch-provable AllPass. An ExecutorRun hook folds dropped rows into es_processed so command tags stay truthful — top-level statements only, since CTE-deleted rows never count toward the outer tag in PostgreSQL.

P2.5 — tombstones-as-rows (DELETE-only fast layer). Qualifying DELETEs append (segment_id, row_offset) tombstone rows to a per-partition _tombstones companion instead of decomposing. Scans subtract tombstones on every read path: decode paths AND them into the selection vector (sequential, Top-N, text, and parallel-worker paths — workers load the tombstone map under the shared parallel snapshot), DeltaXCount counts live rows, and DeltaXMinMax/DeltaXAgg carry stale-plan guards that error (forcing a replan) if tombstones appear after planning. Tombstones are never visible to user queries. Compaction folds tombstoned segments in; the catalog max_segment_id high-water mark is raised before any meta delete so compaction can never reuse a decomposed/dropped segment's id (the shared blob/decompressed caches and the colstats cache are keyed by (companion_oid, segment_id, ...) — id reuse would poison them).

Why / numbers

Requirement: "the user should almost not notice it's not normal postgres by wait time" for DELETE.

• Tombstone DELETE: 0.53 ms vs 0.42 ms on the plain-PostgreSQL twin table — user-indistinguishable latency.
• The same statements under decompose-on-write: 118–173× slower.
• UPDATE uses decompose-on-write — transactional and correct, but slower; closing that gap is the remaining P3 work (the honest blocker: a tombstone-fast UPDATE must still materialize old row versions into the heap for SET expressions / triggers / RETURNING, which costs the same full-column decode the decompose performs).

Design

dev/docs/COMPRESSED_DML.md (shipped/updated in this stack) — §5 decompose-on-write, §5.4 whole-segment DELETE, the P2.5 tombstone fast-layer section, and the P2 deviations list (AccessExclusive partition lock + meta-row delete-first claim protocol; the decompose-cap GUC is deferred to P3).

Testing

• make build — clean.
• make clippy — zero warnings.
• make test (pgrx unit tests) — green on PG 17 and PG 18 (575 each; the PG 18 run covers the version-gated ExecutorRun hook).
• make integration-test PG_VERSIONS=17 — green (396 tests), including the new tests/test_compressed_dml.py (31 tests: decompose UPDATE/DELETE across query shapes, twin-table equality, rollback, concurrency, tombstone fast path, reads-with-tombstones on every query shape, tombstone rollback, compaction folding with segment-id high-water mark, mixed tombstones + heap tail, repeated DELETE accumulation, CTE DELETE ... RETURNING, CTE command-tag integrity, transition-table observation).

Provenance

Extracted from a longer perf-session branch (perf/clickhouse-gap-session); this PR carves out exactly the DML P2 + P2.5 work. Sibling features from that session (dual-mode segment files #32, partition bloom sentinels #35, count sidecars #31) are separate PRs; where their interaction rules matter for this code, comments flag them explicitly as not yet on this branch.