Skip to content

transform/subslice: Add subslice transform#14835

Open
jlucovsky wants to merge 2 commits intoOISF:mainfrom
jlucovsky:7672/16
Open

transform/subslice: Add subslice transform#14835
jlucovsky wants to merge 2 commits intoOISF:mainfrom
jlucovsky:7672/16

Conversation

@jlucovsky
Copy link
Contributor

Continuation of #14824

The subslice transform creates a slice of the input buffer.

Specify the subslice desired -- nbytes and truncate are optional:
        subslice: offset <,nbytes> <,truncate>

offset: Specifies the starting offset for the new subslice. When
negative, expresses how far from the end of the input buffer to begin.
When nbytes is *not* specified, start must be > 0.

nbytes: Specifies the size of the subslice. When negative, specifies the
byte count preceding the offset to include. Nbytes must be > 0.

When nbytes is not specified, the size of the subslice will be the size
of the input buffer - offset.

truncate: Specify behavior when offset + nbytes exceeds buffer length.
When present, trims nbytes such that offset + nbytes equals buffer
length. When not present, an empty buffer is produced.

Examples:
subslice: 1; - The subslice will be a copy of the input
buffer but omit the input buffer's first byte
"This is Suricata" -> "his is Suricata"
subslice: 0, 13; - The slice is created from the first 13 bytes
of the input buffer
"This is Suricata" -> "This is Suric"
subslice: 10, -5; - This is the same as subslice[5, 5]
"This is Suricata" -> "is Su"
subslice: -3; - The subslice will be the last 3 bytes of the
input buffer.
"This is Suricata" -> "ata"

Link to ticket: https://redmine.openinfosecfoundation.org/issues/7672

Describe changes:

  • Add subslice transform and unit tests
  • Document subslice transform, with examples

Updates:

  • Removed support for bracketed values, e.g., [3], [3, 8]`
  • Changed function DetectTransformSubsliceData to be attributed with repr(C)
  • Modified handling of negative nbyte values to mean "bytes from the end" .
  • start=0 is an error unless nbytes is specified
  • end=0 is always an error.
  • Corrected clippy issues.
  • Added configuration variable subslice.truncate to control behavior when offset + nbytes > length
  • Updated s-v branch.
  • Fixed s-v failing tests due to default config settings
  • Rebase
  • Fixed race condition in unittests wrt global variable
  • Removed the truncate global option and replaced with a per-usage truncate option.
  • Rebase
  • Doc update to remove lingering global truncate option.
  • Removed commit that adjusted asn1 underline characters.
  • Updated Rust unit tests to eliminate is_some usage.
  • Removed inadvertent inclusion of Cargo.lock.in
  • Applied truncate option to negative offsets/byte count exceeding buffer length.
  • Reworked slice copy to handle input/output pointing to the same buffer.
  • Expanded documentation to include discussion on negative offset/nbytes exceeding buffer length.
  • Fixed clippy issue
  • Addressed review comments (see transform/subslice: Add subslice transform #14768)
  • Rebase (see transform/subslice: Add subslice transform #14824)

Provide values to any of the below to override the defaults.

  • To use a Suricata-Verify or Suricata-Update pull request,
    link to the pull request in the respective _BRANCH variable.
  • Leave unused overrides blank or remove.

SV_REPO=
SV_BRANCH=OISF/suricata-verify#2749
SU_REPO=
SU_BRANCH=

Issue: 7672

The subslice transform creates a slice of the input buffer.

Specify the subslice desired -- nbytes and truncate are optional:
        subslice: offset <,nbytes> <,truncate>

offset: Specifies the starting offset for the new subslice. When
negative, expresses how far from the end of the input buffer to begin.
When nbytes is *not* specified, start must be > 0.

nbytes: Specifies the size of the subslice. When negative, specifies the
byte count preceding the offset to include. Nbytes must be > 0.

When nbytes is not specified, the size of the subslice will be the size
of the input buffer - offset.

truncate: Specify behavior when offset + nbytes exceeds buffer length.
When present, trims nbytes such that offset + nbytes equals buffer
length. When not present, an empty buffer is produced.

Examples:
        subslice: 1;     - The subslice will be a copy of the input
            buffer but omits the input buffer's first byte
            "This is Suricata" -> "his is Suricata"
        subslice: 0, 13; - The slice is created from the first 13 bytes
            of the input buffer
            "This is Suricata" -> "This is Suric"
        subslice: 10, -5; - The subslice is created starting at offset 10
            and continues to 5 bytes before the end of the input buffer
            "This is Suricata" -> "r"
        subslice: -3; - The subslice will be the last 3 bytes of the
            input buffer.
            "This is Suricata" -> "ata"
Add documentation for the subslice transform.

Issue: 7672
@suricata-qa
Copy link

Information: QA ran without warnings.

Pipeline = 29777

Copy link
Contributor

@catenacyber catenacyber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work,

CI : ✅
Code : commenting
Commits segmentation : cool, I would squash :-p
Commit messages : As already said in the previous version 🟡 looks like it does not match anymore the behavior like When nbytes is not specified, start must be > 0.
Git ID set : looks fine for me
CLA : you already contributed
Doc update : cool
Redmine ticket : ok
Rustfmt : nit

diff --git a/rust/src/detect/transforms/mod.rs b/rust/src/detect/transforms/mod.rs
index da42ecf38e..b20246d484 100644
--- a/rust/src/detect/transforms/mod.rs
+++ b/rust/src/detect/transforms/mod.rs
@@ -26,6 +26,6 @@ pub mod dotprefix;
 pub mod hash;
 pub mod http_headers;
 pub mod strip_whitespace;
+pub mod subslice;
 pub mod urldecode;
 pub mod xor;
-pub mod subslice;

Tests : cool
Dependencies added: none

The length of the input buffer is ``17`` bytes; ``5`` bytes from the end
is ``12``::

subslice: 10, -5;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is ri ?

zlib_deflate; content:"This is compressed then base64-encoded"; startswith; endswith;
sid:2; rev:1;)

subslice
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jufajardini could you take a second look at the docs ?

// offset, nbytes
let nbytes = second.parse::<isize>().ok()?;
if nbytes == 0 {
return None;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we log a user-friendly error that nbytes=0 is not accepted

let nbytes = second.parse::<isize>().ok()?;

if !third.eq_ignore_ascii_case("truncate") {
return None;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same : would we log friendly errors before returning None ?

}
};

// Normalize if indices reversed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can indices get reversed ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

3 participants