v0.7.0

TL;DR
RSigma v0.7.0 is the "any log format" release. The evaluation engine now operates on a generic Event trait instead of raw JSON, a new rsigma-runtime library crate decouples the streaming pipeline from the CLI, and the daemon can ingest JSON, syslog (RFC 3164/5424), logfmt, CEF, and plain text, with auto-detection by default. Hand-rolled zero-dependency parsers for logfmt and CEF keep the dependency tree lean.

This release is inspired by sigma_engine, thanks to @thomaspatzke and Sigma HQ folks.

What's New

Generic Event trait (breaking)

The rsigma-eval::Event struct has been replaced by an Event trait with three concrete implementations:

• JsonEvent: wraps serde_json::Value (the previous behavior)
• KvEvent: key-value map for structured formats (syslog, logfmt, CEF)
• PlainEvent: raw text for keyword-only matching

An EventValue enum provides typed access to field values across all implementations. This is a breaking change: callers using Event::new(value) should switch to JsonEvent::borrow(&value) or JsonEvent::owned(value).

rsigma-runtime crate

The streaming pipeline has been extracted from the CLI daemon into a reusable library crate:

• RuntimeEngine: wraps Engine + CorrelationEngine with rule loading, hot-reload, and state management.
• LogProcessor: batch processing pipeline with ArcSwap for atomic engine swap, pluggable MetricsHook, and EventFilter for JSON payload extraction (e.g. .records[]).
• Input format adapters (input/ module): JSON, syslog, logfmt, CEF, plain text, and auto-detect. Each adapter parses a raw line into EventInputDecoded, a static-dispatch enum that implements Event without dyn overhead.
• I/O primitives: EventSource trait and Sink enum (stdin, stdout, file, NATS) moved from the CLI.

Multi-format input (--input-format)

The daemon and eval commands now accept --input-format and --syslog-tz:

# Auto-detect (default): tries JSON → syslog → plain
rsigma daemon -r rules/

# Explicit syslog with timezone offset
rsigma daemon -r rules/ --input-format syslog --syslog-tz +0530

# logfmt (requires logfmt feature)
rsigma eval -r rules/ --input-format logfmt < app.log

# CEF (requires cef feature)
rsigma eval -r rules/ --input-format cef < arcsight.log

Auto-detect validates syslog parsing results (checks for facility/severity/hostname) before accepting and it won't misparse random text as syslog.

Zero-dependency parsers

• logfmt: hand-rolled parser supporting quoted values with escape sequences, bare keys, and mixed whitespace. No external dependencies.
• CEF (Common Event Format): hand-rolled parser for the full ArcSight CEF spec including 7-field pipe-delimited header + key=value extensions with \=, \n, \\ escapes. Handles syslog-wrapped CEF via find_cef_start().

Both are feature-gated (logfmt, cef) and thoroughly tested with real-world log samples.

Examples and benchmarks

• examples/jsonl_stdin.rs: read NDJSON from stdin, print detections.
• examples/tail_syslog.rs: read a syslog file, parse and evaluate.
• Throughput benchmark suite (runtime_throughput): Criterion benchmarks for the LogProcessor pipeline across all formats.

Baseline results (Apple M4 Pro, 100 rules):

Format	Throughput
Plain text	5.5–10.9 Melem/s
Syslog	1.26–1.40 Melem/s
JSON	955 Kelem/s–1.15 Melem/s
Auto-detect	~966 Kelem/s–1.09 Melem/s

Rule-count scaling is near-flat from 100 to 1,000 rules thanks to logsource index pruning.

Other changes

• Custom attributes (custom_attributes): propagate custom rule attributes through results, then unified across detection and correlation rules into a single custom_attributes field (breaking: custom_rule_attributes removed). Thanks to @fwosar (#26).
• Lint --exclude: glob patterns to skip files during linting, plus detection of deprecated aggregation syntax.
• Line feeds in conditions: fixed parsing of condition expressions containing line breaks. Thanks to @fwosar (#24).
• Dependencies: notify 7 → 8.2, rustls-webpki → 0.103.13.
• Architecture diagram: updated in README to reflect the runtime layer and Event trait.
• BENCHMARKS.md: documents all benchmark groups, baseline results, and the 5% regression threshold.

Breaking Changes

Before (0.6.0)	After (0.7.0)
use rsigma_eval::Event; (struct)	use rsigma_eval::event::Event; (trait)
Event::new(value)	JsonEvent::borrow(&value) or JsonEvent::owned(value)
Event::from_value(v)	JsonEvent::borrow(&v)
result.custom_rule_attributes	result.custom_attributes

Contributors

Thanks to @fwosar for their contributions to this release (#24, #26).

Full Changelog

v0.6.0...v0.7.0