Crawl Budget Waste Estimator
Gives a rough waste share and crawl time from counts of good, parameter, facet and duplicate URLs and your daily crawl rate.
Use the tool
Why this tool exists
On large sites with filters and tracking parameters, most crawlable URLs are variations of a few real pages. If crawlers visit URLs evenly, most of their requests go to waste and new pages wait longer to be found.
Enter your numbers, for example the daily request count from the Crawl Stats report, and see the waste share and how much faster a pass over your real pages could be. It is a model for ranking problems, not a prediction; crawlers favour popular and fresh pages.
How to use it
- Count your real pages and the waste URL types (a crawl export helps).
- Enter your crawler requests per day.
- Read the waste share and time to crawl.
- Fix the biggest waste source first.
Frequently asked questions
Does crawl budget matter for my small site?
Mostly not. It matters for large sites or ones that publish many pages quickly.
Where do I get the numbers?
Crawl a staging copy, use the URL parameter analyzer, and read Crawl Stats in Search Console.
Is the waste share exact?
No. It assumes even visits, which real crawlers do not do.
How do I cut waste?
Canonical tags, fewer internal links to filters, and robots.txt rules for parameters.