# AI Crawlers - Explicitly Allowed User-agent: GPTBot Allow: / User-agent: OAI-SearchBot Allow: / User-agent: OAI-AdsBot Allow: / User-agent: ChatGPT-User Allow: / User-agent: ClaudeBot Allow: / User-agent: anthropic-ai Allow: / User-agent: Google-Extended Allow: / User-agent: PerplexityBot Allow: / User-agent: Perplexity-User Allow: / User-agent: Applebot-Extended Allow: / User-agent: CCBot Allow: / User-agent: Amazonbot Allow: / # Block low-quality scrapers User-agent: Bytespider Disallow: / User-agent: meta-externalagent Disallow: / # Google crawlers (Merchant Center free listings + Shopping ads) — must reach ?wmc-currency= feed landing pages (added 2026-07-18) # Google reads only its own group and ignores "User-agent: *", so the other Disallows are repeated here; only wmc-currency is deliberately left open. User-agent: Googlebot User-agent: Googlebot-Image User-agent: AdsBot-Google Allow: / Disallow: /wp-content/uploads/wc-logs/ Disallow: /wp-content/uploads/woocommerce_transient_files/ Disallow: /wp-content/uploads/woocommerce_uploads/ Disallow: /*?add-to-cart= Disallow: /*?*add-to-cart= Disallow: /*?wc-ajax= Disallow: /*?*wc-ajax= Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php User-agent: * Content-Signal: search=yes, ai-input=yes, ai-train=no Allow: / Disallow: /wp-content/uploads/wc-logs/ Disallow: /wp-content/uploads/woocommerce_transient_files/ Disallow: /wp-content/uploads/woocommerce_uploads/ Disallow: /*?add-to-cart= Disallow: /*?*add-to-cart= # Cache-bypassing / no-SEO-value query URLs (added 2026-07-16 after bot-swarm OOM incident) # wc-ajax = WooCommerce cart fragments (AJAX, never indexed); wmc-currency = Multi Currency variants (duplicate content) Disallow: /*?wc-ajax= Disallow: /*?*wc-ajax= Disallow: /*?wmc-currency= Disallow: /*?*wmc-currency= Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php # Throttle bots that honor Crawl-delay (Bing/Yandex/etc.); Google ignores this and is managed in Search Console Crawl-delay: 10 Sitemap: https://synopower.club/sitemap_index.xml