Agent skill

sitemap-robots

Automated sitemap generation for all locale URLs, robots.txt configuration, and llms.txt for AI crawler optimization. Use when setting up sitemap.xml, configuring crawling rules, or improving discoverability for search engines and AI systems.

Stars 163
Forks 31

Install this agent skill to your Project

npx add-skill https://github.com/majiayu000/claude-skill-registry/tree/main/skills/data/sitemap-robots

SKILL.md

Sitemap & Robots

Sitemap Generation

tsx
// app/sitemap.ts
import type { MetadataRoute } from 'next';
import { locales } from '@/i18n.config';
import { getServices, getProgrammes } from '@/lib/data';

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const siteUrl = process.env.NEXT_PUBLIC_SITE_URL!;
  
  // Static pages
  const staticPages = [
    '',
    '/about',
    '/services',
    '/book',
    '/pricing',
    '/programmes',
    '/testimonials',
    '/faq',
    '/policies',
    '/contact',
  ];

  // Generate URLs for all locales and static pages
  const staticUrls = locales.flatMap((locale) =>
    staticPages.map((page) => ({
      url: `${siteUrl}/${locale}${page}`,
      lastModified: new Date(),
      changeFrequency: page === '' ? 'weekly' : 'monthly' as const,
      priority: page === '' ? 1 : 0.8,
      alternates: {
        languages: Object.fromEntries(
          locales.map((loc) => [loc, `${siteUrl}/${loc}${page}`])
        ),
      },
    }))
  );

  // Dynamic programme pages
  const programmes = await getProgrammes('en');
  const programmeUrls = locales.flatMap((locale) =>
    programmes.map((programme) => ({
      url: `${siteUrl}/${locale}/programmes/${programme.slug}`,
      lastModified: new Date(programme.lastUpdated),
      changeFrequency: 'monthly' as const,
      priority: 0.7,
      alternates: {
        languages: Object.fromEntries(
          locales.map((loc) => [
            loc,
            `${siteUrl}/${loc}/programmes/${programme.slug}`,
          ])
        ),
      },
    }))
  );

  // Dynamic service pages (if you have individual service pages)
  const services = await getServices('en');
  const serviceUrls = locales.flatMap((locale) =>
    services.map((service) => ({
      url: `${siteUrl}/${locale}/services/${service.slug}`,
      lastModified: new Date(service.lastUpdated),
      changeFrequency: 'monthly' as const,
      priority: 0.7,
      alternates: {
        languages: Object.fromEntries(
          locales.map((loc) => [
            loc,
            `${siteUrl}/${loc}/services/${service.slug}`,
          ])
        ),
      },
    }))
  );

  return [...staticUrls, ...programmeUrls, ...serviceUrls];
}

Robots.txt

tsx
// app/robots.ts
import type { MetadataRoute } from 'next';

export default function robots(): MetadataRoute.Robots {
  const siteUrl = process.env.NEXT_PUBLIC_SITE_URL!;
  
  return {
    rules: [
      {
        userAgent: '*',
        allow: '/',
        disallow: [
          '/api/',
          '/checkout/',
          '/_next/',
        ],
      },
    ],
    sitemap: `${siteUrl}/sitemap.xml`,
  };
}

LLMs.txt for AI Crawlers

txt
// public/llms.txt
# Studio Name - Pilates & Yoga Coaching

## About
Professional Pilates and Yoga coaching studio offering personalized 1:1 sessions, 
small group classes, and digital programmes. Evidence-informed approach focused on 
strength, mobility, posture, and stress relief.

## Languages
- Portuguese (pt-PT) - Default
- English (en)
- Turkish (tr)
- Spanish (es)
- French (fr)
- German (de)

## Important Pages
- Home: /[locale]/
- About: /[locale]/about
- Services: /[locale]/services
- Pricing: /[locale]/pricing
- Programmes: /[locale]/programmes
- Book: /[locale]/book
- FAQ: /[locale]/faq
- Contact: /[locale]/contact

## Services Offered
- 1:1 Pilates Sessions (mat and reformer)
- 1:1 Yoga Sessions
- Small Group Classes
- Online Coaching
- Digital Programmes (on-demand video courses)

## Booking
Sessions can be booked via the /book page. Available in-person and online.

## Contact
Email: hello@studioname.com
Location: Istanbul, Turkey

## Disclaimer
Content is educational and not medical advice. Consult a qualified professional 
before starting any exercise programme.

Extended LLMs-Full.txt

tsx
// scripts/generate-llms-full.ts
// Run this as a build step to generate comprehensive content

import { locales } from '@/i18n.config';
import { getServices, getProgrammes, getFAQs } from '@/lib/data';
import fs from 'fs';

async function generateLlmsFull() {
  let content = `# Studio Name - Complete Content Reference\n\n`;
  content += `Generated: ${new Date().toISOString()}\n\n`;

  // Services
  content += `## Services\n\n`;
  const services = await getServices('en');
  for (const service of services) {
    content += `### ${service.name}\n`;
    content += `${service.longDesc}\n`;
    content += `- Duration: ${service.duration}\n`;
    content += `- Price from: €${service.priceFrom}\n`;
    content += `- Delivery: ${service.delivery}\n`;
    content += `- Tags: ${service.tags.join(', ')}\n\n`;
  }

  // Programmes
  content += `## Digital Programmes\n\n`;
  const programmes = await getProgrammes('en');
  for (const programme of programmes) {
    content += `### ${programme.title}\n`;
    content += `- Level: ${programme.level}\n`;
    content += `- Duration: ${programme.durationText}\n`;
    content += `- Access: ${programme.accessLengthText}\n`;
    content += `- Equipment: ${programme.equipment.join(', ')}\n`;
    content += `- Outcomes: ${programme.outcomes.join('; ')}\n`;
    content += `- Price: €${programme.price}\n\n`;
  }

  // FAQs
  content += `## Frequently Asked Questions\n\n`;
  const faqs = await getFAQs('en');
  for (const group of faqs) {
    content += `### ${group.category}\n\n`;
    for (const item of group.items) {
      content += `**Q: ${item.question}**\n`;
      content += `A: ${item.answer}\n\n`;
    }
  }

  fs.writeFileSync('public/llms-full.txt', content);
  console.log('Generated llms-full.txt');
}

generateLlmsFull();

Package.json Script

json
{
  "scripts": {
    "generate:llms": "tsx scripts/generate-llms-full.ts",
    "build": "npm run generate:llms && next build"
  }
}

Verification

After deployment, verify:

  • https://yoursite.com/sitemap.xml - All locale URLs present
  • https://yoursite.com/robots.txt - Sitemap reference correct
  • https://yoursite.com/llms.txt - Human-readable summary
  • Google Search Console - Submit sitemap

Expand your agent's capabilities with these related and highly-rated skills.

Didn't find tool you were looking for?

Be as detailed as possible for better results