Overview

Overview

Let’s face it: manual recon is a time-sink. Spider Suite was built to handle the heavy lifting. It’s a powerful crawler that actually understands modern web architecture, giving you a readable map of a target’s entire footprint. If you’re hunting bugs or pentesting a complex app, it gives you the bird’s-eye view you need to spot vulnerabilities that others miss.

Requirements

Note

Spider Suite runs on 64-bit machines only and is currently available for Windows and Linux operating systems. It does not support x32 systems.

Features

Crawlers

SpiderSuite includes five specialized crawler types:

• Standard Crawler - Best for static pages and traditional websites
• Headless Crawler - Perfect for JavaScript-heavy SPAs
• Bruteforce Crawler - Directory and file discovery
• Onion/TOR Crawler - Specialized for darkweb crawling
• Links Crawler - Extract and crawl links from various file formats

Testing

• Intercept - Man-in-the-middle proxy for HTTP(S) traffic
• Request - Custom HTTP request builder
• Fuzzer - Automated fuzzing with multiple techniques

Tools

• Decoder Tool - A tool for encoding/decoding, encrypting/decrypting, compressing/decompressing data. Many encryption, encoding and compressor algorithms are supported.
• Compare Tool - A tool for comparing different webpages. The comparison shows similarities and differences between the webpages.

Import

Import http(s) webpages obtained from other tools. Supported formats and Tools includes:

• HTTP Archive (.har) - HTTP archive format that is supported by all web browsers
• Web Archive (.warc & .warc.gz) - Web Archive format that is supported by most web archiving sites e.g. archive.org
• Katana (.json & .txt) - Katana crawler export files
• Caido (.csv & .json) - Caido proxy export files
• Burpsuite (.xml) - BurpSuite tool export files
• Fiddler (.saz) - Fiddler proxy export files
• SpiderSuite (.json/.jsonl/.xml) - Spider Suite's own export file types

Export

Export http(s) webpages (content or links) to different storage formats. Supported formats includes:

• JSON (.json & jsonl) - Stores the webpage's requests and responses to json or jsonl filetypes
• XML (.xml) - Stores the webpage's requests and responses to xml filetype
• HTML (.html) - Stores the webpage's URL, status, size, content-type and html title to a html filetype
• Filesystem - Save the webpages to the filesystem. Each webpage is saved as a distinct file on the filesystem
• Sitemap (.xml) - Saves the webpages into a sitemap file which appropriate location and last modification parameters
• HTTP Archive (.har) - Saves the webpage's requests and responses into a http archive file
• Web Archive (.warc / .warc.gz) - Saves the webpage's requests and responses into a web archive file
• LLM Ready (.json/.jsonl) - Processes the HTML and JSON webpages into LLM-ready data then stores the data into JSON file

HTTPQL Filter

HTTPQL is a query language designed by the team at Caido for filtering and analyzing HTTP traffic. The HTTPQL syntax used by Spider Suite has been modified to suit the application's provided features.