Cloudflare has launched a new policy framework called the Content Signals Policy, aimed at giving website owners and content creators more control over how their content is used by AI systems. The move is seen as a direct challenge to Google’s AI Overviews, which draw answers from web content without always linking back to original sources. At the heart of the initiative is a new licensing mechanism that allows websites to explicitly grant or deny access to AI crawlers. Unlike traditional robots.txt, which relies on voluntary compliance, Cloudflare’s policy adds legal weight by signaling that website preferences are not just suggestions but binding terms. This could have significant implications for companies that scrape the web at scale. Cloudflare CEO Matthew Prince emphasized the growing imbalance in how AI companies access content. While firms like OpenAI use separate bots for search and AI training, Google combines its search crawler with its AI answer engines, giving it a unique advantage. “Every AI answer engine should have to play by the same rules,” Prince said. “Google combines its crawler for search with its AI answer engines, which gives them a unique and unfair advantage.” The new policy will automatically apply to over 3.8 million domains using Cloudflare’s infrastructure—roughly 20% of the web—starting Wednesday. Website owners can now set granular preferences: allowing or blocking access for search, AI input (such as chatbot responses), or AI training (the foundational data used to build models). This gives creators the ability to block AI bots from scraping content used to train models, a growing concern among publishers and creators who feel their work is being used without consent or compensation. Prince argued that the policy could carry legal consequences if ignored, especially for a company like Google, which has a large stake in AI and search. Google maintains that its AI Overviews still drive traffic to websites and improve the overall health of the web. Executives have said they respect content creators and aim to support the open internet. However, critics argue that the current model benefits big tech at the expense of smaller publishers. Cloudflare’s approach strengthens the outdated robots.txt standard by turning it into a more enforceable tool. It also encourages transparency and accountability in how AI companies collect data. As AI continues to evolve, the ability for creators to control their digital assets may become a defining issue of the web’s future. Prince concluded that the goal is to ensure the web remains open, fair, and sustainable. “The internet cannot wait for a solution while in the meantime, creators' original content is used for profit by other companies,” he said.