Crawler policy

A bounded crawler with visible rules.

LucidCrawl/0.4 (+https://lucidcrawl.com/crawler) requests public pages only, respects crawl policy, revalidates redirects, and blocks private or reserved networks.

Every audit enforces accepted page, depth, byte, redirect, render, and time limits. Exclusions remain visible in the evidence receipt.