Faceted navigation

Filters such as size, colour and price that create new URLs for every combination — a common source of crawl waste.

Every combination of filters can generate a crawlable URL, so a modest catalogue can produce millions of near-duplicate pages. Google's guidance is to keep low-value combinations out of crawling — for example with robots.txt rules or by not making them links — and to let only valuable filtered pages be indexed.