xml-sitemap

What this skill does

# SEO Technical: Sitemap

Guides sitemap creation, auditing, and optimization for search engine discovery.

**When invoking**: On **first use**, if helpful, open with 1–2 sentences on what this skill covers and why it matters, then provide the main output. On **subsequent use** or when the user asks to skip, go directly to the main output.

## Scope (Technical SEO)

- **Sitemap**: Create XML sitemap; submit to Google Search Console
- **URL discovery**: Help search engines find pages; especially important for large sites or poor internal linking

## Task

Generate an XML Sitemap that complies with the sitemaps.org protocol from the project's page list, and declare it in robots.txt.

## Initial Assessment

**Check for project context first:** If `.claude/project-context.md` or `.cursor/project-context.md` exists, read it for site URL and page structure.

Identify:
1. **Site URL**: Base domain (e.g., `https://example.com`)
2. **URL count**: Total indexable pages (single sitemap vs. sitemap index)
3. **Data source**: Static config, CMS, file system, or hybrid

### Precondition: Does the site need a sitemap?

Before generating, assess whether a sitemap is warranted. A sitemap is most valuable when:
- **Large site** (hundreds+ pages) or **new site** with few backlinks
- **Deep or orphaned pages** that internal links don't reach well
- **Rich media** (images, videos, news) needing extension metadata
- GSC shows growing "Discovered – not indexed" or "Not discovered" counts

Small sites (< 50 pages) with strong internal linking may not strictly need one, but creating a sitemap has near-zero cost and provides future-proof infrastructure. If in doubt, err on the side of creating.

## 1. Protocol Essentials

| Item | Spec |
|------|------|
| Single sitemap limit | 50,000 URLs, 50MB (uncompressed) |
| Sitemap index | When exceeding limit, split and have main index reference sub-sitemaps |
| Encoding | UTF-8 |
| URL format | Full URL, same host, include `https://` |
| Required tags | `<loc>` |
| Optional tags | `<lastmod>`, `<changefreq>`, `<priority>` |

## 2. Field Requirements

| Field | Description | Recommendation |
|-------|--------------|---------------|
| url | Full URL | `https://example.com/path` |
| lastModified | Page last modified time | Use page metadata, ISO 8601; use `YYYY-MM-DD` or omit when no data |
| changeFrequency | Update frequency | Home `daily`, list pages `weekly`, content pages `monthly` |
| priority | Relative importance | Home 1.0, aggregate pages 0.9, content pages 0.7–0.8, others 0.5–0.6 |

### lastmod (Critical)

- **Must be accurate**: Reflect actual page modification time, not sitemap generation time. Google requires verifiability; Bing reports ~18% of sitemaps have incorrect lastmod values.
- **Format**: W3C Datetime (`YYYY-MM-DD` or `YYYY-MM-DDTHH:MM:SS+TZD`), e.g. `2025-01-15`, `2025-01-15T14:30:00+08:00`.
- **Avoid**: Using `new Date()` for lastmod—causes all URLs to share the same timestamp; search engines may ignore.
- **Apply when**: Content updates, structured data changes, or important link changes.

### changefreq / priority

- **changefreq**: Hints only; does not directly determine crawl frequency. Values: `always`, `hourly`, `daily`, `weekly`, `monthly`, `yearly`, `never`.
- **priority**: 0.0–1.0; **does not affect ranking**; set higher for important pages; avoid identical values for all.

## 3. Architecture & Split

### Single Sitemap

- When URLs >50,000, generate `/sitemap.xml` directly.

### Sitemap Index (Multiple Sub-sitemaps)

- When exceeding limit, split by type or language; main index references sub-sitemaps.
- Example splits: `/sitemap/posts.xml`, `/sitemap/pages.xml`, `/sitemap/zh.xml`, `/sitemap/en.xml`.
- Main index outputs `/sitemap.xml` or `/sitemap-index.xml`, each entry as `<sitemap><loc>...</loc></sitemap>`.

### Multilingual Sites

- Split by locale: `/sitemap/zh.xml`, `/sitemap/en.xml`.
- Or by content type + language: `/sitemap/zh-posts.xml`, `/sitemap/en-posts.xml`.

### Multi-Language Sitemap (hreflang in Sitemap)

For multilingual sites, add `xhtml:link` hreflang alternates inside each `<url>` entry. **Recommended for large sites** (100+ multilingual pages); centralizes hreflang management.

**Rules**:
- Every language version must link to ALL others, including itself (self-reference).
- Include `x-default` pointing to default locale.
- Use `xmlns:xhtml="http://www.w3.org/1999/xhtml"` namespace.
- `<loc>` typically uses default-locale (clean) URL; `x-default` points there too.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:xhtml="http://www.w3.org/1999/xhtml">
  <url>
    <loc>https://example.com/page</loc>
    <xhtml:link rel="alternate" hreflang="en" href="https://example.com/page" />
    <xhtml:link rel="alternate" hreflang="zh" href="https://example.com/zh/page" />
    <xhtml:link rel="alternate" hreflang="x-default" href="https://example.com/page" />
  </url>
</urlset>
```

List all language sitemaps in sitemap index; include in robots.txt.

## 4. Implementation

| Tech Stack | Implementation |
|------------|-----------------|
| Next.js App Router | `app/sitemap.ts` export `MetadataRoute.Sitemap` or `generateSitemaps` |
| Next.js Pages Router | `pages/sitemap.xml.ts` or `getServerSideProps` return XML |
| Astro | `src/pages/sitemap-index.xml.ts` or `@astrojs/sitemap` |
| Vite / Static build | Build script generates `public/sitemap.xml` |
| Other | Generate static `/sitemap.xml` or return dynamically via API |

### Route Exclusion

- If the project has i18n / middleware redirects, exclude sitemap paths to avoid redirect.
- Example (Next.js matcher): `'/((?!api|_next|sitemap|sitemap-index|.*\\..*).*)'`.

## 5. Page Scope

### Include

- Home: `/`
- Locale/region home pages (e.g. `/zh`, `/en`)
- All indexable content pages, list pages, category pages

### Exclude

- `/api/*`, `/admin/*`, `/_next/*`
- Static assets (images, JS, CSS, etc.). For image discovery, use **image sitemap** extension—see **image-optimization**. For video discovery, use **video sitemap** extension—see **video-optimization**
- Login, admin, drafts, and other pages not intended for indexing

## 6. Data Source & Maintenance (Single Source of Truth)

- **Single source of truth**: Read URL list from config, CMS, or metadata; avoid hardcoding in sitemap.
- **Multiple page types**: Tools, blog, marketing pages can be merged into one array for unified generation.
- **New pages**: Add only to data source; sitemap updates automatically; avoid maintaining multiple places.

### Central Config (Recommended)

Create a config (e.g., `site-pages-config.ts`) that exports:
- Page slugs/paths by section (tools, blog, marketing, etc.)
- Optional: `modifiedDate` per page for accurate lastmod
- Function: `getAllPageUrls(baseUrl)` for sitemap and IndexNow

**Why**: Sitemap, IndexNow, and feed can all import from the same config—no duplicate URL maintenance. IndexNow should use the same URL list; avoid separate hardcoded lists.

## 7. robots.txt

Add to robots.txt:

```
Sitemap: https://example.com/sitemap.xml
```

With multiple sitemaps, only declare the main index.

## 8. Output Format

### Single Sitemap Example

```xml
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2025-01-15</lastmod>
    <changefreq>daily</changefreq>
    <priority>1.0</priority>
  </url>
  <url>
    <loc>https://example.com/page</loc>
    <lastmod>2025-01-10</lastmod>
    <changefreq>weekly</changefreq>
    <priority>0.8</priority>
  </url>
</urlset>
```

### Sitemap Index Example

```xml
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <sitemap>
    <loc>https://example.com/sitemap/pages.xml</loc>
    <lastmod>2025-01-15</lastmod>
  </sitemap>
  <sitemap>
    <loc>https://example.com/sitemap/posts.xml</loc>
    <lastmod>2025-01