# Othor AI, robots.txt # https://othor.ai # Every LLM and AI crawler is explicitly allowed. This is deliberate. # Updated 12 September 2026 # --------------------------------------------------------------- # Default # --------------------------------------------------------------- User-agent: * Allow: / Disallow: /api/ Disallow: /_next/static/chunks/ Disallow: /*?*utm_ Disallow: /*?*ref= # --------------------------------------------------------------- # OpenAI # --------------------------------------------------------------- User-agent: GPTBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ChatGPT Agent Allow: / User-agent: OAI-SearchBot Allow: / # --------------------------------------------------------------- # Anthropic # --------------------------------------------------------------- User-agent: ClaudeBot Allow: / User-agent: Claude-Web Allow: / User-agent: Claude-User Allow: / User-agent: Claude-SearchBot Allow: / User-agent: anthropic-ai Allow: / # --------------------------------------------------------------- # Google, AI Overviews, Gemini, Vertex # --------------------------------------------------------------- User-agent: Googlebot Allow: / User-agent: Google-Extended Allow: / User-agent: GoogleOther Allow: / User-agent: GoogleOther-Image Allow: / User-agent: GoogleOther-Video Allow: / User-agent: GoogleAgent-Mariner Allow: / User-agent: Google-CloudVertexBot Allow: / User-agent: CloudVertexBot Allow: / # --------------------------------------------------------------- # Perplexity # --------------------------------------------------------------- User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / # --------------------------------------------------------------- # Meta # --------------------------------------------------------------- User-agent: Meta-ExternalAgent Allow: / User-agent: Meta-ExternalFetcher Allow: / User-agent: meta-externalagent Allow: / User-agent: meta-externalfetcher Allow: / User-agent: FacebookBot Allow: / # --------------------------------------------------------------- # Apple # --------------------------------------------------------------- User-agent: Applebot Allow: / User-agent: Applebot-Extended Allow: / # --------------------------------------------------------------- # Microsoft, Bing, Copilot # --------------------------------------------------------------- User-agent: Bingbot Allow: / User-agent: BingPreview Allow: / User-agent: msnbot Allow: / # --------------------------------------------------------------- # Other AI crawlers and training sets # --------------------------------------------------------------- User-agent: MistralAI-User Allow: / User-agent: MistralAI-User/1.0 Allow: / User-agent: CCBot Allow: / User-agent: cohere-ai Allow: / User-agent: cohere-training-data-crawler Allow: / User-agent: Diffbot Allow: / User-agent: Bytespider Allow: / User-agent: Amazonbot Allow: / User-agent: YouBot Allow: / User-agent: Timpibot Allow: / User-agent: Ai2Bot Allow: / User-agent: Ai2Bot-Dolma Allow: / User-agent: DuckAssistBot Allow: / # --------------------------------------------------------------- # Sitemaps and machine-readable summary # --------------------------------------------------------------- Sitemap: https://othor.ai/sitemap.xml