URL List Normalizer and Deduplicator

Makes equal URLs look equal so you can count, deduplicate or group them. Choose which differences to ignore, such as tracking parameters, parameter order, www and trailing slashes.

Use the tool

Why this tool exists

Crawls, logs and analytics exports contain the same page many times in slightly different forms. Before you compare lists or count unique pages you need one form for each URL.

The tool lowercases the scheme and host, drops default ports, resolves dot segments, uppercases percent-encoding and decodes unreserved characters (RFC 3986 normalization). The other steps are your choice, because they depend on your server: for example, paths are case-sensitive on most servers, so lowercase paths is off by default.

How to use it

  1. Paste your URLs, one per line.
  2. Tick the normalization steps you want.
  3. Choose the output: normalized list, unique list or duplicate groups.
  4. Check the groups before you redirect or delete anything.

Frequently asked questions

Are URLs case-sensitive?

The scheme and host are not. The path usually is, and the query often is. That is why lowercasing the path is optional.

Which parameters count as tracking?

utm_*, gclid, gbraid, wbraid, fbclid, msclkid, dclid, yclid, mc_cid, mc_eid, igshid, _ga, _gl, ref_src and twclid.

Is sorting parameters safe?

Usually, but a few sites depend on the order. Check one example.

Does it follow redirects?

No. It only rewrites the text you paste.