Redirect Hits in Logs (URLs Bots Keep Requesting That Redirect)

Lists redirecting URLs by number of bot hits, with status codes, bots and last seen time.

Use the tool

Why this tool exists

A redirect costs the crawler one request and gives it nothing to index. If bots hit the same redirecting URL day after day, something still links to it: a menu, a sitemap, an old backlink or an internal page.

The log records the status but not the redirect target, so the tool cannot show full chains. It ranks the sources instead. Take the top URLs, check where they lead with a redirect checker, and update the links that point to them. Temporary redirects that are hit often are flagged for review.

How to use it

  1. Paste access log lines.
  2. Choose the bots and the minimum number of hits.
  3. Read the ranked list of redirecting URLs.
  4. Find and update internal links, sitemap entries and menus that point to them.

Frequently asked questions

Why can it not show redirect chains?

The default log format has no Location header. Only the status of each request is logged.

Is a 302 a problem?

A temporary redirect that lasts for months is usually meant to be permanent. Review it.

Do external links matter?

Yes. Old backlinks keep bots coming to redirected URLs. Those cannot be changed, which is fine as long as the redirect works.

Can I trust the bot names in the result?

Only as claims. The tool reads the user-agent text, and anyone can send a fake one. For anything that matters, check the IP addresses against the operator's published ranges or with reverse DNS.