Website owners make money from human views, so AI crawlers need to pay their fair share, too.
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works .
(Image credit: Getty Images / Sundry Photography) Cloudflare CEO Matthew Prince said that his company has blocked over 416 billion AI bot requests since making it the default option in July of this year after it announced the Content Independence Day initiative. Prince said in an interview with Wired that this feature allows website owners to block AI crawlers by default, unless the AI company pays them to gain access to their content.
“The business model of the internet has always been to generate content that drive traffic and then sell either things, subscriptions, or ads, Prince told Wired . “What I think people don’t realize, though, is that AI is a platform shift. The business model of the internet is about to change dramatically. I don’t know what it’s going to change to, but it’s what I’m spending almost every waking hour thinking about.”
While Cloudflare blocks almost all AI crawlers, there’s one particular bot it cannot block without affecting its customers’ online presence — Google . The search giant combined its search and AI crawler into one, meaning users who opt out of Google’s AI crawler won't be indexed in Google search results. “You can’t opt out of one without opting out of both, which is a real challenge — it’s crazy,” Prince continued. “It shouldn’t be that you can use your monopoly position of yesterday in order to leverage and have a monopoly position in the market of tomorrow.”
AI adoption far outpaces that of the early Internet — report sheds light on worldwide AI penetration and usage patterns
Anthropic targets gigantic $26 billion in revenue by the end of 2026
Key considerations
- Investor positioning can change fast
- Volatility remains possible near catalysts
- Macro rates and liquidity can dominate flows
Reference reading
- https://www.tomshardware.com/tech-industry/big-tech/SPONSORED_LINK_URL
- https://www.tomshardware.com/tech-industry/big-tech/cloudflare-says-it-has-fended-off-416-billion-ai-bot-scrape-requests-in-five-months-ceo-warns-of-dramatic-shift-for-internet-business-model#main
- https://www.tomshardware.com
- Senators lobby for SAFE Chips Act, which would curb leading-edge AI chip exports to China — proposed bill would restrict AMD and Nvidia to H20/MI308-class accel
- Scythe Big Shuriken 4 Review: Compact SFF cooling for Mini-ITX
- NVIDIA and AWS Expand Full-Stack Partnership, Providing the Secure, High-Performance Compute Platform Vital for Future Innovation
- This MSI Pro 27-inch gaming monitor plummets to its lowest-ever price of $89.99 at Newegg — 1080p, 144Hz display is a gaming and productivity double-threat
- Intel admits it needs more Core Ultra 200-series wafers — 'If we had more Lunar Lake wafers, we would be selling more Lunar Lake'
Informational only. No financial advice. Do your own research.