Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 48 additions & 0 deletions docs/modules.md
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,54 @@ both 32-bit forms are accepted, so values from shodan or any favicon-hash tool
drop in without conversion. pair it with a `status: 200` matcher so an error
page served for `/favicon.ico` is not hashed. a finding fires when the body
hashes to any listed value.
### dsl matcher

evaluate one or more boolean expressions against the response. the syntax and
variable names are nuclei's, so an expression written for a nuclei template
pastes in unchanged.

```yaml
matchers:
- type: dsl
dsl:
- "status_code == 200 && contains(body, 'admin')"
- "content_length > 1024"
```

the variables bound for every expression:

| variable | type | value |
|----------|------|-------|
| `status_code` | int | response status code |
| `body` | string | response body, after the 5 MB cap |
| `content_length` | int | length of `body` in bytes |
| `header` / `all_headers` | string | the response headers, one `Name: value` per line |
| `duration` | float | round-trip time in seconds |
| `host` | string | `host[:port]` of the request url |

nuclei's capitalized url-part variables are bound as well, generated by nuclei's
own helper so the semantics match a pasted template exactly: `BaseURL`,
`RootURL`, `Hostname` (host with port), `Host` (host without port), `Port`,
`Path`, `File`, `Scheme`, `Query`, plus the dns parts `FQDN`, `RDN`, `DN`, `TLD`
and `SD`.

named extractor values are bound too, so an expression can test something an
extractor pulled out of the same response; in a request chain it also sees the
variables earlier steps extracted. an extractor whose name collides with a
builtin shadows it, matching nuclei's mutation order.

helper functions are an allowlist, not a blocklist: string inspection and
transforms, `regex`/`regex_all`/`regex_any`, base64/hex/url/html encode and
decode, `md5`/`sha1`/`sha256`/`mmh3`, and numeric conversion. anything else
fails at load, so a dependency bump cannot quietly introduce a helper that reads
files, makes its own requests, or allocates without bound. expressions are also
capped at 4096 bytes.

expressions are compiled and checked when the module loads, so a typo fails the
module up front instead of silently never matching. at match time an expression
that errors or yields a non-boolean counts as a miss. multiple expressions
combine with AND by default, or with `condition: or`.

### combining matchers

multiple matchers are combined with AND logic by default.
Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ require (
github.com/charmbracelet/log v1.0.0
github.com/gocolly/colly/v2 v2.3.0
github.com/likexian/whois v1.15.7
github.com/projectdiscovery/dsl v0.8.20
github.com/projectdiscovery/goflags v0.1.74
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e
github.com/projectdiscovery/nuclei/v3 v3.11.0
github.com/projectdiscovery/retryabledns v1.0.115
github.com/projectdiscovery/utils v0.11.1
Expand Down Expand Up @@ -272,7 +274,6 @@ require (
github.com/projectdiscovery/blackrock v0.0.1 // indirect
github.com/projectdiscovery/cdncheck v1.2.42 // indirect
github.com/projectdiscovery/clistats v0.1.4 // indirect
github.com/projectdiscovery/dsl v0.8.20 // indirect
github.com/projectdiscovery/fastdialer v0.5.11 // indirect
github.com/projectdiscovery/fasttemplate v0.0.2 // indirect
github.com/projectdiscovery/freeport v0.0.7 // indirect
Expand All @@ -282,7 +283,6 @@ require (
github.com/projectdiscovery/goja_nodejs v0.0.0-20260618132410-8519f75f703d // indirect
github.com/projectdiscovery/gologger v1.1.71 // indirect
github.com/projectdiscovery/gostruct v0.0.2 // indirect
github.com/projectdiscovery/govaluate v0.0.0-20260615100919-5ee2581bbf7e // indirect
github.com/projectdiscovery/gozero v0.1.1-0.20260530071156-fa1dad563d76 // indirect
github.com/projectdiscovery/hmap v0.0.101 // indirect
github.com/projectdiscovery/httpx v1.9.0 // indirect
Expand Down
189 changes: 189 additions & 0 deletions internal/modules/dsl.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
/*
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
: :
: █▀ █ █▀▀ · Blazing-fast pentesting suite :
: ▄█ █ █▀ · BSD 3-Clause License :
: :
: (c) 2022-2026 vmfunc, xyzeva, :
: lunchcat alumni & contributors :
: :
·━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━·
*/

package modules

import (
"fmt"
"net/http"
"net/url"
"strings"
"sync"

"github.com/projectdiscovery/dsl"
"github.com/projectdiscovery/govaluate"
nucleiutils "github.com/projectdiscovery/nuclei/v3/pkg/protocols/utils"
)

// maxDSLExprLen bounds a single dsl expression. govaluate's Evaluate takes no
// context and cannot be interrupted mid-helper, so capping the input is the
// reliable defense against a pathological expression.
const maxDSLExprLen = 4096

// allowedDSLHelpers is the set of helper functions a dsl expression may call,
// keyed by the underscore-stripped name so both alias forms (to_lower and
// tolower) resolve. It is an allowlist so a future projectdiscovery/dsl bump
// cannot silently reintroduce a side-effecting helper (llm_prompt, public_ip,
// wait_for, the gadget generators) or an unbounded-allocation one (repeat, the
// rand_* and faker families): anything unnamed here fails to compile.
var allowedDSLHelpers = func() map[string]bool {
names := []string{
// string inspection / comparison
"contains", "contains_all", "contains_any", "starts_with", "ends_with",
"line_starts_with", "line_ends_with", "equals_any", "len", "index",
// string transforms
"to_lower", "to_upper", "trim", "trim_left", "trim_right", "trim_space",
"trim_prefix", "trim_suffix", "split", "join", "replace", "replace_regex",
"concat", "reverse",
// regex (RE2, linear-time)
"regex", "regex_all", "regex_any",
// encode / decode
"base64", "base64_decode", "hex_encode", "hex_decode",
"url_encode", "url_decode", "html_escape", "html_unescape",
// hashing / fingerprint
"md5", "sha1", "sha256", "mmh3",
// numeric / conversion
"to_number", "to_string",
}
m := make(map[string]bool, len(names))
for _, n := range names {
m[stripUnderscore(n)] = true
}
return m
}()

func stripUnderscore(s string) string { return strings.ReplaceAll(s, "_", "") }

// dslHelpers is the curated govaluate function map: every entry of
// dsl.HelperFunctions() whose (underscore-stripped) name is allowlisted.
var dslHelpers = func() map[string]govaluate.ExpressionFunction {
all := dsl.HelperFunctions()
out := make(map[string]govaluate.ExpressionFunction, len(all))
for name, fn := range all {
if allowedDSLHelpers[stripUnderscore(name)] {
out[name] = fn
}
}
return out
}()

// dslCache memoizes compiled expressions by source string. A compiled
// *EvaluableExpression is safe to Evaluate concurrently (value receiver, pooled
// scratch state), so sharing one across goroutines is fine.
var dslCache sync.Map // string -> *govaluate.EvaluableExpression

// hasNonEmptyDSL reports whether exprs holds at least one non-empty expression.
func hasNonEmptyDSL(exprs []string) bool {
for _, e := range exprs {
if strings.TrimSpace(e) != "" {
return true
}
}
return false
}

// dslCompile compiles expr against the curated helpers, caching the result, so
// an over-length expression, a syntax error or a non-allowlisted function is
// rejected at module load rather than silently missing at match time.
func dslCompile(expr string) (*govaluate.EvaluableExpression, error) {
if len(expr) > maxDSLExprLen {
return nil, fmt.Errorf("dsl expression exceeds %d bytes", maxDSLExprLen)
}
if cached, ok := dslCache.Load(expr); ok {
return cached.(*govaluate.EvaluableExpression), nil
}
compiled, err := govaluate.NewEvaluableExpressionWithFunctions(expr, dslHelpers)
if err != nil {
return nil, fmt.Errorf("dsl expression %q: %w", expr, err)
}
actual, _ := dslCache.LoadOrStore(expr, compiled)
return actual.(*govaluate.EvaluableExpression), nil
}

// dslVars builds the variable environment a dsl expression evaluates against,
// using nuclei's lowercase names so nuclei dsl expressions paste in unchanged.
// Named extractor values overlay the builtins (matching nuclei's mutation
// order), so an extractor may reference, and on a name clash shadow, a builtin.
func dslVars(mc *MatchContext) map[string]interface{} {
headers := getPart("header", mc.Resp, mc.Body)
vars := map[string]interface{}{
"status_code": statusCodeOf(mc.Resp),
"body": mc.Body,
"content_length": len(mc.Body),
"all_headers": headers,
"header": headers,
"duration": mc.Duration.Seconds(),
"host": hostOf(mc.URL),
}
// the capitalized nuclei url-part vars, plus the dns ones. nuclei's own
// generator keeps them faithful: Host is the hostname without port while
// Hostname carries it, Path is the directory, Port defaults by scheme.
for k, v := range nucleiutils.GenerateVariables(mc.URL, false, nil) {
vars[k] = v
}
for k, v := range mc.Extracted {
vars[k] = v
}
return vars
}

// hostOf returns the host[:port] of a request URL, matching nuclei's `host`
// variable so a pasted nuclei expression (host == "example.com") behaves as
// expected. On an unparseable URL it falls back to the raw string rather than
// binding an empty host.
func hostOf(rawURL string) string {
if u, err := url.Parse(rawURL); err == nil && u.Host != "" {
return u.Host
}
return rawURL
}

func statusCodeOf(resp *http.Response) int {
if resp == nil {
return 0
}
return resp.StatusCode
}

// evalDSL folds a dsl matcher's expressions under its condition (default AND).
// An expression that errors at eval, or yields a non-bool, counts as false
// (fail-closed), matching the engine's swallow-at-match-time invariant.
func evalDSL(m *Matcher, mc *MatchContext) bool {
if len(m.DSL) == 0 {
return false
}
vars := dslVars(mc)
or := strings.EqualFold(m.Condition, "or")
for _, expr := range m.DSL {
matched := evalOneDSL(expr, vars)
if or && matched {
return true
}
if !or && !matched {
return false
}
}
return !or
}

func evalOneDSL(expr string, vars map[string]interface{}) bool {
compiled, err := dslCompile(expr)
if err != nil {
return false // unreachable after load validation; fail closed anyway
}
result, err := compiled.Evaluate(vars)
if err != nil {
return false // unbound var / type mismatch -> miss
}
b, ok := result.(bool)
return ok && b
}
Loading
Loading