Cloudflare, Inc.

New York Stock Exchange
Back to all articles
Bullish +75

Cloudflare Helps End the Search-or-AI-Training Tradeoff

AI-generated analysis

🚀 Cloudflare launches new controls allowing publishers to refuse AI training while staying fully in search results.

📊 Mixed-use crawlers now account for 36.6% of verified crawler traffic on Cloudflare's network.

✅ Apple, Google, and Microsoft are labeled as 'Accountable' for meeting criteria or setting timelines for compliance.

⚙️ The company replaces the single 'Block AI Bots' switch with three independent controls for search, training, and agents.

🤖 New websites receive tailored recommended settings, such as blocking AI agents on ad-carrying pages by default.

🔄 Cloudflare introduces 'Bot Preference Sync' to apply crawling preferences automatically across all supported crawlers.

📜 Operators must meet four specific requirements to earn the 'Accountable' designation for AI crawling.

🔍 Google's 'Google-Extended' and other separate crawler strategies allow Cloudflare to block training without affecting search.

🌐 Cloudflare is participating in IETF standards work, including the development of the 'ai-prefs' specification.

Bullish Signals
  • Cloudflare successfully resolves the long-standing tradeoff between search discoverability and AI training control for website owners.
  • The company secures 'Accountable' status from major industry leaders like Apple, Google, and Microsoft, validating its new criteria.
  • Mixed-use crawlers now represent a significant 36.6% of traffic on Cloudflare's network, highlighting the relevance of these new controls.
  • The introduction of 'Bot Preference Sync' simplifies management for customers by applying preferences automatically across all crawlers.
Full Analysis
Cloudflare, Inc. (NYSE: NET) announced today the launch of new controls and recommended settings that allow publishers to independently manage how their content is used for search, AI training, and AI agents. This initiative introduces an 'Accountable' designation for AI crawlers, enabling website owners to refuse AI training while remaining fully discoverable in traditional search results. The company reports that mixed-use crawlers now constitute 36.6% of verified crawler traffic on its network. Previously, refusing AI training often meant losing search visibility from these crawlers; the new settings resolve this tradeoff by allowing site owners to block training without affecting search indexing. Major tech giants including Apple, Google, and Microsoft have been labeled as 'Accountable' for meeting specific criteria or providing timelines to do so. Cloudflare is replacing its previous 'Block AI Bots' switch with three independent controls: one for search, one for AI training, and one for AI agents. New websites on the platform will see tailored recommended settings based on their business model, such as disallowing AI training for ad-carrying sites while keeping search crawling active. The company also introduces 'Bot Preference Sync' to simplify managing these preferences across all supported crawlers. To qualify for the 'Accountable' designation, crawler operators must meet four requirements, including providing clear opt-out mechanisms for AI training and summaries, offering URL-level visibility into content usage, and publicly confirming that opting out of training will not impact search rankings. Cloudflare states it is actively participating in IETF standards work to support these emerging preferences.