Skip to content
Browse all guides

Issue #215 · important

Issue 215

What is this issue?

This issue checks whether URLs with multiple filter and sort parameters (faceted navigation combination pages) have proper SEO directives to prevent them from being indexed as separate pages.

A URL passes this check if it has at least one of the following:

  • A canonical tag pointing to a URL with fewer parameters (preferably the base URL)
  • A noindex directive in the HTML meta tags
  • A noindex directive in the HTTP response headers (X-Robots-Tag)

A URL fails if it:

  • Contains two or more filter/sort parameters without proper handling
  • Has a self-referencing canonical that includes all the same parameters
  • Has more than five query parameters without any handling directive

Example:

  • Base URL: https://example.com/shoes
  • Combination URL: https://example.com/shoes?color=red&size=42&brand=nike&sort=price
  • Passing: The page has <meta name="robots" content="noindex, follow">
  • Failing: The page has no canonical or noindex directive, or has a self-referencing canonical

Why does it matter?

Faceted navigation creates severe SEO problems when left unhandled:

  • Crawlability: Faceted navigation can generate thousands or even millions of URL combinations from a single base page. Search engine crawlers waste their entire crawl budget on these low-value pages, leaving important pages uncrawled.

  • Indexability: Near-duplicate combination pages enter the search index, causing index bloat and reducing the overall quality signal of the site.

  • Rankings: Link equity and ranking signals are diluted across hundreds of near-duplicate faceted URLs instead of consolidating on the canonical page.

  • Duplicate content: Multiple faceted URLs with substantially similar content confuse search engines about which version to rank, potentially causing all versions to be suppressed.

  • Crawl budget waste: Googlebot and other crawlers allocate a finite number of requests per site. Unhandled faceted navigation causes crawlers to spend their budget on low-value combination pages.

Resolving this issue improves the overall SEO health score by preventing index bloat, optimizing crawl budget usage, and ensuring search engines focus on indexing valuable pages.

How to fix it

  1. Audit all faceted URL combinations. Use server access logs or a full site crawl to collect all URLs containing two or more query parameter keys. Group them by parameter count and combination type to understand the scope.

  2. Define an indexability policy for faceted URLs. Decide which facet combinations have unique SEO value and should be indexed. In most cases, only these are worth indexing:

    • Single-facet pages that correspond to queries users actually search for (e.g., /shoes?brand=nike if "nike shoes" has significant search demand)
    • The base category page without parameters

  3. Apply noindex to all non-valuable combination pages. For every combination page that doesn't meet your indexability policy, add to the <head>:

    <meta name="robots" content="noindex, follow" />

    Use follow so links on the page are still crawled even if the page itself is excluded from the index.

  4. Add canonical tags pointing to cleaner URLs. For combination pages that should consolidate to a simpler version:

    • Point to the base URL (no parameters) for maximum consolidation
    • Or point to a single-facet URL if that has SEO value
    • Ensure the canonical URL has fewer parameters than the page URL

  5. Avoid self-referencing canonicals on combination pages. A canonical that includes all the same parameters as the page URL provides no deduplication benefit.

  6. Handle excessive parameter depth. If a URL has more than five query parameters, it requires immediate noindex or canonical handling regardless of the specific parameters used.

  7. Normalize parameter order. Ensure faceted navigation generates consistent URL formats regardless of which filter the user selects first (e.g., /shoes?color=red&size=42 should always appear in the same order).

  8. Verify the fix via re-crawl. After implementing directives, re-crawl the affected faceted URLs to confirm the tags are present and correctly formed.

Examples

Example 1: Basic Faceted Navigation with Noindex

Scenario: An e-commerce category page with multiple filter combinations.

Problematic state (failing):

URL: https://example.com/shoes?color=red&size=42&brand=nike
Canonical: None
Noindex: None

Corrected state (passing):

URL: https://example.com/shoes?color=red&size=42&brand=nike
Noindex: <meta name="robots" content="noindex, follow" />

Example 2: Faceted Navigation with Canonical to Base URL

Scenario: A product listing page with sort and filter parameters.

Problematic state (failing):

URL: https://example.com/shoes?sort=price_asc&color=blue
Canonical: <link rel="canonical" href="https://example.com/shoes?sort=price_asc&color=blue" />

Corrected state (passing):

URL: https://example.com/shoes?sort=price_asc&color=blue
Canonical: <link rel="canonical" href="https://example.com/shoes" />

Example 3: Excessive Parameter Depth

Scenario: A URL with more than five query parameters creating combinatorial risk.

Problematic state (failing):

URL: https://example.com/products?color=red&size=42&brand=nike&sort=price&material=leather&style=casual
Canonical: None
Noindex: None

Corrected state (passing):

URL: https://example.com/products?color=red&size=42&brand=nike&sort=price&material=leather&style=casual
Noindex: <meta name="robots" content="noindex, follow" />

How PixyScan detects this

PixyScan follows a logical detection process to identify unhandled faceted navigation pages:

  1. URL parameter analysis: The system extracts all query parameters from each crawled URL and counts the total number of distinct parameter keys.

  2. Faceted combination identification: A URL is classified as a faceted combination page when it meets either condition:

    • Contains two or more parameter keys from known facet patterns (filter parameters like color, size, brand; or sort parameters like sort, order)
    • Contains one filter key and one sort key simultaneously (e.g., ?color=red&sort=price_asc)

  3. Parameter depth check: The system checks whether the URL contains more than five distinct query parameter keys, which is considered excessive and high-risk regardless of pattern matching.

  4. Header inspection: The crawler checks the HTTP X-Robots-Tag response header for the presence of noindex (case-insensitive).

  5. HTML parsing: The raw HTML <head> section is parsed to:

    • Locate the first <link rel="canonical"> tag and extract its href attribute
    • Check for <meta name="robots"> tags containing the noindex token
    • Detect if multiple canonical tags are present (a misconfiguration)

  6. Canonical evaluation: If a canonical URL is found:

    • It is normalized (relative URLs are resolved against the page's base URL)
    • The system checks whether the canonical URL contains fewer parameter keys than the page URL
    • It verifies whether the canonical is self-referencing (identical to the page URL including all parameters)

  7. Handling determination: A faceted combination page is considered "handled" if at least one condition is met:

    • Noindex header is present
    • Noindex meta tag is present
    • Canonical is present, is not self-referencing, and points to a URL with fewer parameters

  8. Issue triggering: An issue is raised when a faceted combination page lacks proper handling (no valid canonical or noindex directive).

References