User-agent: * Disallow: /admin Disallow: /front_css User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: Claude-SearchBot Allow: / # User-triggered fetcher — activated when a Claude user requests a specific page User-agent: Claude-User Allow: / # Primary indexing crawler — builds Perplexity's AI search index User-agent: PerplexityBot Allow: / # User-triggered retrieval — fetches pages when a Perplexity user clicks a citation User-agent: Perplexity-User Allow: / # Controls Gemini model training and Vertex AI data collection User-agent: Google-Extended Allow: / # Gemini-specific fetch for AI answer grounding User-agent: Googlebot-Extended Allow: / User-agent: Meta-ExternalAgent Allow: / User-agent: CCBot Allow: / Sitemap: https://www.parboiler.com/sitemap.xml