What Happened: Cloudflare's AI Bot Blocking May Also Block Googlebot

Cloudflare's new AI bot blocking feature, designed to stop AI training bots from scraping your content, may also be blocking legitimate search engine bots like Googlebot and Bingbot. A Redditor in the r/SEO community reported that enabling Cloudflare's "AI Training = Block" setting caused Googlebot and Bingbot to receive HTTP 403 errors when fetching their sitemap. This is a critical issue because if search engines can't crawl your site, you won't appear in search results, and that can kill your organic traffic.

Does This Affect Your Business?

If you use Cloudflare and have enabled AI bot blocking, your site could be at risk of being de-indexed from Google and Bing. Even if you haven't enabled it yet, Cloudflare has announced that starting September 15, 2026, it will automatically block mixed-usage bots—bots that both index for search and collect data for AI training—on new domains. That means Googlebot and Bingbot, which are classified as "Search + Training" bots, could be blocked by default. If you rely on organic search traffic, this is a direct threat to your bottom line.

What This Means for Your Business

If you're a business owner who uses Cloudflare and has enabled AI bot blocking, you need to check your settings immediately. The Redditor's experience shows that the block can affect your sitemap, which is the first thing search engines fetch. If they can't access your sitemap, they may not index your pages, and you'll lose visibility in search results. Even if you haven't enabled the block, you should be aware of the upcoming default change in September 2026. If you're not using Cloudflare, this doesn't affect you directly, but it's a reminder to check your own bot management settings.

Your Move: Check Your Cloudflare Settings Now

Log into your Cloudflare dashboard and go to the "AI Crawlers & Scrapers" section. If you have "AI Training" set to "Block," consider disabling it or adjusting your settings to allow Googlebot and Bingbot. You can also use Cloudflare's "Bot Fight Mode" settings to ensure legitimate search bots are not blocked. If you're unsure, test by fetching your sitemap using Google Search Console's URL inspection tool. If you see 403 errors, you need to fix your settings immediately.

Bottom Line

Cloudflare's AI bot blocking is a double-edged sword. It protects your content from AI scraping, but it can also block search engines, which is far more damaging. The good news is that you have control over your settings. Act now to ensure your site remains visible in search results.



Source: Search Engine Journal

FAQ

Yes, if it blocks Googlebot from crawling your site, your pages may not be indexed, leading to a drop in rankings and organic traffic.

Disable the 'AI Training' block in your Cloudflare settings, or ensure that Googlebot and Bingbot are allowed in your bot management rules.