Pre-compiled IR rule packs

[mostafa]

Status

Parked: marginal value. Reconsidered after the base IR shipped.

A pack would store the HIR (IrRule), not compiled rules, because CompiledRule holds Regex / Aho-Corasick automata / IpNet that are not serializable (the same reason the HIR restart cache stores HIR). So a loader still runs compile_to_compiled per rule: compile is the expensive step, and a pack does not skip it. A pack only saves YAML parse + pipeline + lower, which are already millisecond-scale, while plaintext YAML stays portable, diffable, git-friendly, and signable. The "cut startup cost" motivation does not hold up.

An additional finding: re-expressing pipeline transforms on the HIR is partially ill-defined, because the faithful HIR resolves modifiers (contains/base64/cased/re/...) into matcher variants and discards the raw modifier list that several transforms gate on. Pipelines are architecturally a pre-lowering concern.

Revisit trigger

Pick this up only when a concrete consumer justifies the complexity of a versioned binary format:

• a WASM / embedded eval target that must not bundle a YAML parser;
• a cross-engine binary interchange need not covered by the portable-IR protobuf work;
• a distribution requirement that specifically needs a single content-addressed binary rather than a signed YAML bundle.

Prototype (unmerged)

A tested prototype exists on a local branch: PackHeader + Pack over CBOR (magic + schema-version check, header-only inspect), Engine::save_pack / load_pack, convert_pack, and filter injection on the HIR (apply_filters), all with round-trip parity tests. Pipeline-on-IR, a pack CLI, CI publishing, and the dynamic-source variant were not built.

Original proposal (superseded by the status above)

Summary

Serialize compiled IR rule packs (static and dynamic-source-aware) for fast load and as the artifact format pack distribution builds on.

Motivation

Large corpora pay parse/compile cost at every process start. Precompiled packs were meant to cut startup and give distribution a stable media type. (See Status: the compile cost is not actually skipped.)

Proposed approach

• Serialization of the IR with header metadata (CBOR was chosen over postcard because LogSource's flattened map is unknown-length).
• Load path that verifies version compatibility.
• Dynamic-source-aware pack variant once the daemon-scoped source cache exists.

Out of scope

• Registry push/pull (separate distribution item).
• Opaque encrypted packs.

References

• IR serialization (the HIR cache: Engine::save_hir / load_hir).
• Daemon source cache.