Skip to content
Browse all guides

Issue #171 · important

Issue 171

What is this issue?

Separate URLs per Language (Subdirectory or ccTLD, Not Just Cookies)

This issue checks whether your website uses distinct, crawlable URLs for each language or regional version instead of relying solely on cookies, JavaScript redirects, or session-based language switching that hide content variations behind a single URL.

What This Issue Checks

The audit verifies that each language or regional version of a page is accessible at a distinct URL using:

  • Subdirectories (/en/, /fr/)
  • Subdomains (en.example.com, fr.example.com)
  • Country-code top-level domains (ccTLDs like .fr, .de)

And does NOT rely solely on:

  • Cookies to switch languages
  • JavaScript redirects based on browser language
  • Session-based language detection
  • Query string parameters only (e.g., ?lang=fr)

What is Considered a Passing Implementation

Your website passes this check when:

  • Each language version has its own distinct, crawlable URL
  • Hreflang alternate URLs point to distinct URL paths or domains
  • Language switching doesn't rely solely on cookies or JavaScript
  • The URL structure clearly indicates the language/region (subdirectory, subdomain, or ccTLD)

Real-World Example

Passing Implementation (Subdirectory approach):

https://example.com/en/about/    ← English version
https://example.com/fr/about/    ← French version
https://example.com/es/about/    ← Spanish version

Failing Implementation (Cookie-based switching):

https://example.com/about/        ← Same URL for all languages
  Cookie: lang=fr                  ← Language controlled by cookie

Why does it matter?

Impact on SEO

Crawlability

Google's crawler is a server-side bot that fetches raw HTTP responses. It does not execute JavaScript, set cookies, or maintain session state between requests. If your website uses cookies or client-side JavaScript to serve different language versions at the same URL, Google can only ever index one version — typically the default.

Indexability

When all language versions are served at the same URL, search engines cannot differentiate between them. They index only the default language version, and all other language variants become invisible to search engines.

Rankings

Users in different regions searching in their own language won't find your content if it's not indexed under distinct URLs. This severely limits your international SEO reach and rankings in target markets.

User Experience

While not directly an SEO factor, proper URL structure for languages ensures users can:

  • Bookmark specific language versions
  • Share links to specific language versions
  • Navigate directly to their preferred language version

Duplicate Content

Serving all languages from the same URL doesn't create duplicate content issues, but it does create "invisible content" issues — search engines only see one language version, making all other language content effectively invisible.

AI Search / AEO

As AI-powered search becomes more prevalent, AI systems need to understand which language version to reference when answering queries. Distinct URLs for each language help AI systems correctly identify and reference the appropriate language version.

How Resolving This Issue Improves SEO Health Score

Fixing separate URLs per language issues improves your overall SEO health score by:

  • Ensuring all language versions are crawlable and indexable
  • Allowing search engines to serve the correct language version to users in different regions
  • Improving international SEO reach and rankings
  • Strengthening hreflang effectiveness (hreflang requires distinct URLs to work properly)
  • Making your multilingual content fully accessible to search engines

How to fix it

Practical Recommendations

1. Use Google-Recommended URL Structures

Implement one of these URL structures for multilingual content:

Subdirectory (Recommended for most sites):

https://example.com/en/about/
https://example.com/fr/about/
https://example.com/es/about/

Subdomain:

https://en.example.com/about/
https://fr.example.com/about/
https://es.example.com/about/

ccTLD (Strongest geo-targeting signal):

https://example.com/about/    (for US/English)
https://example.fr/about/     (for France/French)
https://example.de/about/     (for Germany/German)

2. Avoid Cookie-Based Language Switching

Don't rely solely on cookies to switch languages:

Wrong Approach:

GET /about/ HTTP/1.1
Cookie: lang=fr

HTTP/1.1 200 OK
Content-Type: text/html
<!-- Content served in French based on cookie -->

Google's crawler never sends language cookies, so it only indexes the default (cookie-absent) version.

3. Avoid JavaScript-Only Language Detection

Don't rely solely on JavaScript to detect and redirect to the correct language:

Wrong Approach:

<script>
  if (navigator.language.startsWith("fr")) {
    window.location.href = "/fr/";
  }
</script>

The crawler does not execute JavaScript, so the redirect never fires during crawling.

4. Use Hreflang with Distinct URLs

Ensure your hreflang tags point to distinct URLs for each language variant:

<link rel="alternate" hreflang="en" href="https://example.com/en/about/" />
<link rel="alternate" hreflang="fr" href="https://example.com/fr/about/" />
<link rel="alternate" hreflang="es" href="https://example.com/es/about/" />

5. If Using Query Parameters, Ensure They Create Distinct URLs

Query parameters can work but are not Google's preferred approach:

Acceptable but not optimal:

https://example.com/about/?lang=en
https://example.com/about/?lang=fr

Better approach:

https://example.com/en/about/
https://example.com/fr/about/

Priority Order

  1. First: Implement distinct URLs for each language version (subdirectory recommended)
  2. Second: Remove cookie-only or JavaScript-only language switching
  3. Third: Update hreflang tags to point to the new distinct URLs
  4. Fourth: Set up proper redirects from old URLs if migrating from cookie/JS-based switching
  5. Fifth: Ensure all language versions are accessible without requiring cookies or JavaScript

Examples

Example 1: Cookie-Based Language Switching (Fails)

Scenario

A website serves different language versions at the same URL, using cookies to determine which language to display.

Problematic State (Fails)

GET /about/ HTTP/1.1
Host: example.com

HTTP/1.1 200 OK
Set-Cookie: lang=en; Path=/
Content-Type: text/html
<html lang="en">
<!-- English content served based on cookie -->

GET /about/ HTTP/1.1
Host: example.com
Cookie: lang=fr

HTTP/1.1 200 OK
Content-Type: text/html
<html lang="fr">
<!-- French content served based on cookie -->

Result: Google's crawler never sends language cookies, so it only indexes the default (English) version. The French version is invisible to search engines.

Corrected State (Passes)

GET /en/about/ HTTP/1.1
Host: example.com

HTTP/1.1 200 OK
Content-Type: text/html
<html lang="en">
<link rel="alternate" hreflang="en" href="https://example.com/en/about/">
<link rel="alternate" hreflang="fr" href="https://example.com/fr/about/">
<!-- English content -->

GET /fr/about/ HTTP/1.1
Host: example.com

HTTP/1.1 200 OK
Content-Type: text/html
<html lang="fr">
<link rel="alternate" hreflang="en" href="https://example.com/en/about/">
<link rel="alternate" hreflang="fr" href="https://example.com/fr/about/">
<!-- French content -->

Result: Both language versions have distinct, crawlable URLs. Google can index both versions and serve the correct one to users in different regions.

Example 2: JavaScript-Only Language Redirect (Fails)

Scenario

A website uses JavaScript to detect the user's browser language and redirect to the appropriate language version.

Problematic State (Fails)

<!-- On page: https://example.com/about/ -->
<!DOCTYPE html>
<html>
  <head>
    <script>
      if (navigator.language.startsWith("fr")) {
        window.location.href = "/fr/about/";
      } else if (navigator.language.startsWith("es")) {
        window.location.href = "/es/about/";
      }
    </script>
  </head>
  <body>
    <!-- Default English content -->
  </body>
</html>

Result: Google's crawler does not execute JavaScript, so the redirect never fires. Only the default English content is indexed.

Corrected State (Passes)

<!-- On page: https://example.com/en/about/ -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <link rel="alternate" hreflang="en" href="https://example.com/en/about/" />
    <link rel="alternate" hreflang="fr" href="https://example.com/fr/about/" />
    <link rel="alternate" hreflang="es" href="https://example.com/es/about/" />
  </head>
  <body>
    <!-- English content served directly from this URL -->
  </body>
</html>

Result: Each language version is served from its own distinct URL without requiring JavaScript execution.

Example 3: Query Parameter Language Switching (Warning)

Scenario

A website uses query parameters to differentiate language versions.

Problematic State (Not Recommended)

https://example.com/about/?lang=en
https://example.com/about/?lang=fr
https://example.com/about/?lang=es

Result: While crawlable, this approach is not recommended by Google. Query parameters can be confusing for users and may not be as strongly associated with language targeting as subdirectories.

Corrected State (Passes - Recommended)

https://example.com/en/about/
https://example.com/fr/about/
https://example.com/es/about/

Result: Subdirectory approach is Google's recommended URL structure for multilingual content. It's clear, crawlable, and strongly signals language targeting.

How PixyScan detects this

PixyScan checks whether your website uses distinct, crawlable URLs for each language or regional version, rather than relying solely on cookies, JavaScript redirects, or session-based language switching.

Detection Process

PixyScan follows these logical steps to identify language URL structure issues:

1. Fetch the Page Without Language Cookies

PixyScan issues an HTTP GET request to the target URL without any language cookies set. This simulates how Google's crawler fetches pages.

2. Extract Hreflang Alternate URLs

PixyScan parses the <head> section to find all <link rel="alternate" hreflang="..."> tags and extracts their href values to analyze the URL structure.

3. Analyze URL Structure

PixyScan examines the hreflang URLs to determine the language URL structure:

  • Subdirectory: URLs contain language path segments (/en/, /fr/)
  • Subdomain: URLs use language subdomains (en.example.com)
  • ccTLD: URLs use different country-code domains (.fr, .de)
  • Query parameter: URLs use language query parameters (?lang=fr)
  • Cookie-only: No distinct URLs; language controlled by cookies

4. Check for Cookie-Based Language Indicators

PixyScan inspects:

  • The Set-Cookie HTTP response header for language-related cookies
  • Raw HTML for JavaScript that sets language cookies
  • Raw HTML for cookie-based language switching code

5. Check for JavaScript-Based Language Detection

PixyScan scans the raw HTML for JavaScript code that:

  • Detects navigator.language or navigator.userLanguage
  • Performs redirects based on browser language
  • Sets language preferences without distinct URLs

6. Validate Query String Parameters

PixyScan checks if language is switched via query string parameters (e.g., ?lang=fr, ?locale=en) as the primary mechanism.

When the Issue is Flagged

The issue is flagged when any of these conditions are met:

  • Cookie-only language switching: Language switching appears to rely on cookies rather than distinct URLs
  • Query parameter language switching: Language versions are differentiated only by query string parameters (not recommended by Google)
  • No distinct language URLs detected: No distinct URLs were detected for language variants (Google requires each language version to have a unique, crawlable URL)

When the Issue Passes

The issue passes when:

  • Hreflang alternate URLs use distinct URL paths or domains
  • Language variants are distinguished by URL path segment, subdomain, or domain — not by cookie alone
  • The page does not rely solely on cookies or JavaScript for language switching
  • Each language version has its own indexable URL

References