Uncrawled Sitemap URLs Report (Never Requested by Bots in the Log)

Shows which sitemap URLs received no bot request during the log period, grouped by folder, with a coverage percentage.

Use the tool

Why this tool exists

A sitemap tells crawlers what you want crawled. The log tells you what they fetched. URLs that are in the first and absent from the second may have a discovery problem, a quality problem, or just a log that is too short.

The folder table shows where the gaps cluster, which usually points to a template, a weak internal link path or a blocked section. Bots visit rarely used URLs rarely, so a week of logs will list pages that a month of logs would cover. Any status code counts as a request here.

How to use it

  1. Paste access log lines in the first box.
  2. Paste the sitemap XML or a URL list in the second box.
  3. Pick the bots to count.
  4. Read the coverage by folder, then the list of uncrawled URLs.

Frequently asked questions

How long should the log period be?

As long as you can get. Seven days is a minimum. Large sites often need a month to see every URL.

Does an uncrawled URL mean Google ignores it?

No. It may simply not have been due for a visit. Compare with a longer log and with the URL Inspection tool.

Can I trust the bot names in the result?

Only as claims. The tool reads the user-agent text, and anyone can send a fake one. For anything that matters, check the IP addresses against the operator's published ranges or with reverse DNS.

Is my log uploaded?

No. The log is parsed in your browser and never sent anywhere. Logs contain IP addresses, so you may want to remove them first with the log privacy workbench.