httpql

HTTPQL Query Language

HTTPQL is a powerful query language for filtering and searching HTTP requests and responses in SpiderSuite.

Overview

HTTPQL allows you to write expressive queries to find exactly the pages you're looking for. It was originally designed by the team at Caido and has been adapted for SpiderSuite.

Note

SpiderSuite uses a slightly modified version of HTTPQL to match the application's features and data structure. See Caido's HTTPQL documentation at https://docs.caido.io/app/reference/httpql.html.

Syntax

The constructing primitives of an HTTPQL query statement, in order of position, are the:

1. Namespace
2. Field
3. Operator
4. Value

---

Namespaces

Info: Namespaces are project-specific.

Namespace	Description
req	All proxied HTTP requests.
resp	All proxied HTTP responses.

---

Fields

req

Field	Description	Value Type
created_at	The date and time the request was sent.	Date/Time: RFC3339 (2024-06-24T17:03:48+00:00) / ISO 8601 (2024-06-24T17:03:48+0000) / RFC2822 (Mon, 24 Jun 2024 17:03:48 +0000) / RFC7231 (Mon, 24 Jun 2024 17:03:48 GMT) / ISO9075 (2024-06-24T17:03:48Z)
ext	The extension of the requested file.	String/Byte
host	The value of the request's Host header.	String/Byte
method	The HTTP method used for the request.	String/Byte
path	The URL path (includes files).	String/Byte
port	The port of the target server.	Integer
query	The URL query string (excludes the leading ?).	String/Byte
header	The request's headers (returns true if even one header matches)	String/Byte
cookie	The request' cookies (returns true if even one cookie matches).	String/Byte
body	The request body data.	String/Byte
raw	The full raw data of the request (includes request line, headers, and body data).	String/Byte
len	The request size in bytes (includes request line, headers, and body data).	Integer
body_len	The request body's size in bytes	Integer

resp

Field	Description	Value Type
code	The status code of the response.	Integer
raw	The full raw data of the response (includes response line, headers, and body data).	String/Byte
roundtrip	The total request/response cycle time (in milliseconds).	Integer
header	The response's headers (returns true if even one header matches)	String/Byte
cookie	The response' cookies (returns true if even one cookie matches).	String/Byte
len	The response size in bytes (includes response line, headers, and body data).	Integer
body_len	The response body's size in bytes	Integer

---

Operators

Operator	Description	Value Type	Additional Details
eq	Equal to the supplied value.	String/Byte, Integer	Case sensitive. Requires a leading . character for the ext field.
gt	Greater than the supplied value.	Date/Time, Integer
gte	Greater than or equal to the supplied value.	Integer
lt	Less than the supplied value.	Date/Time, Integer
lte	Less than or equal to the supplied value.	Integer
ne	Not equal to the supplied value.	String/Byte, Integer	Case sensitive. Requires a leading . character for the ext field.
cont	Contains the supplied value.	String/Byte	Case insensitive.
ncont	Does not contain the supplied value.	String/Byte	Case insensitive.
regex	Matches to the regular expression.	String/Byte	Rust-flavored syntax.
nregex	Does not match to the regular expression.	String/Byte	Rust-flavored syntax.

---

Combining Statements

Query statements can be combined together using logical operators and logical grouping.

Logical Operators

Operator	Description
AND	Both the left and right clauses must be true.
OR	Either the left or right clause must be true.

Note

Operators are case insensitive. Both AND and OR have the same default priority, but AND takes precedence over OR in evaluation order (see Logical Grouping below).

Logical Grouping

SpiderSuite supports operator precedence: AND has a higher priority than OR.

Expression	Equivalent To
<Clause1> AND <Clause2> OR <Clause3>	((<Clause1> AND <Clause2>) OR <Clause3>)
<Clause1> OR <Clause2> AND <Clause3>	(<Clause1> OR (<Clause2> AND <Clause3>))
<Clause1> AND <Clause2> AND <Clause3>	((<Clause1> AND <Clause2>) AND <Clause3>)

Note

While parentheses are optional, it is recommended to use them to make your logical grouping explicit and clear.

---

Comments

SpiderSuite doesn't support comments in HTTPQL queries.