Fake Googlebot Detector for Access Logs

Lists IPs that use a Google crawler user-agent and, with Google's ranges pasted, flags the ones that do not belong.

Use the tool

Why this tool exists

Scrapers often use the Googlebot user-agent because many sites let it through. Real Googlebot comes from Google's published IP ranges.

Paste your log and Google's JSON ranges and the tool lists every IP that claims to be Google with the verdict. Without ranges it lists the IPs and tells you how to verify them with reverse DNS. Use a fresh copy of the ranges, as a stale file gives false alarms.

How to use it

  1. Paste log lines.
  2. Paste Google's common-crawlers.json or CIDR lines.
  3. Read the range check per IP.
  4. Block fakes at the firewall.

Frequently asked questions

Which Google files?

common-crawlers.json, special-crawlers.json and user-triggered-fetchers.json are listed on Google's verification page.

Without ranges?

You get the IP list and the reverse DNS procedure.

Could a real Googlebot be flagged?

If your range file is outdated, yes.

Is the log uploaded?

No.