robots.txt Allow / Disallow Conflict Detector

Looks for Allow and Disallow rules in the same group that can match the same URL and tells you the winner and the effect.

Use the tool

Why this tool exists

When an Allow and a Disallow both match a URL, the longer pattern wins, and on a tie the Allow wins. That is easy to get wrong by eye, especially with wildcards, and a rule you wrote as an exception may do nothing.

The detector checks each Allow against each Disallow in a group, names the winner and says whether the Allow works as an exception or is overridden. It finds overlaps by testing sample paths built from the patterns, so unusual wildcard mixes can be missed; review the important ones.

How to use it

  1. Paste the robots.txt.
  2. Read each overlapping pair and the winner.
  3. Remove or reorder nothing: order does not matter, only length.
  4. Confirm with the tester.

Frequently asked questions

Who wins a tie?

The Allow rule, as in RFC 9309 and Google's documentation.

Does rule order matter?

No. Specificity (pattern length) decides.

Why might an Allow do nothing?

A longer Disallow that also matches the URL overrides it.

Does it check different user-agents against each other?

No. Only rules in the same effective group are compared.