URL Visit

A small cache warmer script. Minimal tooling. Just curling. For large sitemaps, URL visit supports parallel processing of URLs. This utilizes the --parallel feature of curl.

Usage

Simple crawling

url_visit.sh --url <DOMAIN>

Define the amount of parallel processes

Default is 3. The maximum for parallel workers is 50.

url_visit.sh --url <DOMAIN> --workers=5