Summary
Embed machine-verifiable example events in Sigma rules under rsigma.exemplars (expected match/no-match), verify them with a new rsigma rule test command, lint shape checks, an MCP tool, and ADS validation integration so each rule carries its own regression evidence.
Motivation
Backtest works against external corpora, but per-rule true-positive and true-negative evidence does not travel with the rule. Embedding exemplars makes the rule the single reviewable artifact for Detection-as-Code diffs and CI.
Proposed approach
- Typed extraction in the parser for exemplar event and correlation sequence payloads.
- Static lint for shape and rule-kind mismatches (never executes exemplars).
- Runner in the evaluator that compiles rules and asserts each exemplar outcome.
rsigma rule test with pipeline support, text/json output, and --fail-on-missing for CI.
- MCP
test_exemplars tool; a well-formed expect-match exemplar can satisfy ADS validation when prose is absent.
Out of scope
- Replacing corpus-based
rule backtest.
- Auto-generating exemplars from production traffic (separate capture work).
Tasks
References
- Custom attributes and ADS modules in
rsigma-parser
rule backtest expectations schema for output conventions
- MCP tool-router pattern in
rsigma-mcp
Summary
Embed machine-verifiable example events in Sigma rules under
rsigma.exemplars(expected match/no-match), verify them with a newrsigma rule testcommand, lint shape checks, an MCP tool, and ADS validation integration so each rule carries its own regression evidence.Motivation
Backtest works against external corpora, but per-rule true-positive and true-negative evidence does not travel with the rule. Embedding exemplars makes the rule the single reviewable artifact for Detection-as-Code diffs and CI.
Proposed approach
rsigma rule testwith pipeline support, text/json output, and--fail-on-missingfor CI.test_exemplarstool; a well-formed expect-match exemplar can satisfy ADS validation when prose is absent.Out of scope
rule backtest.Tasks
rsigma rule testCLI and goldensReferences
rsigma-parserrule backtestexpectations schema for output conventionsrsigma-mcp