Cloudflare and OpenAI: Making AI discovery better for everyone

Optimizing your site for AI is of little use if AI can't find it. The answer to someone's question may already be on your website: the installation guide you spent a week writing, or a first-hand account of using a product, with the details that make the difference for whether they buy it. As more people turn to AI for help, you want those systems to find your work, represent it accurately, and send interested people your way. You also want control over how they access your site.

Cloudflare and OpenAI launched a research pilot in July 2026 with those interests in mind. We wanted to find out whether signals from Cloudflare’s network could help ChatGPT users discover new content more efficiently.

Today, we’re sharing early results. So far, the answer is a resounding yes: the pilot has brought previously unknown pages into OpenAI’s search index, giving ChatGPT more sources to draw on to help people find relevant, timely information. Along the way, we’ve learned more about what makes a page worth discovering and how AI systems use the web.

Finding more of the web

The web is full of specifics. Someone fixing a washing machine needs the manual for Continue reading

The Internet has a second audience

For most of its history, the Internet had one audience that paid the bills: people. We read the articles, saw the ads, and bought the subscriptions. Bots were always there, but they were mostly large, automated operations that didn't view ads, pay for anything, or read in any meaningful sense.

That's changing fast. At the end of 2024, Cloudflare handled an average of 63 million HTTP requests a second. Today, it's almost doubled to 115 million, with peaks above 150 million. Over the past year, daily requests from AI agents on our network grew by more than 1,700%. This year, for the first time, more than half of Internet traffic wasn't human.

The human web didn't shrink to make room. A second audience arrived alongside it: agents, software acting on behalf of people. They sit somewhere between humans and traditional bots. They don't respond to ads, but there's usually a person behind them with a job to get done. For businesses that learn to serve them and capture value from them, agents are additive. For those that don't, they're extractive.

What our customers need hasn't changed: to be discovered, to tell great stories, to build great experiences, and to sell. What's Continue reading

From ranking to recommended: get your site ready to thrive in the age of AI agents

Your next customer may not find you through a search engine. Instead, they'll ask an AI assistant: "how do I do X?"; "which option is best for someone like me?"; "just handle it for me" and an agent will find the answer, weigh the options, and act on their behalf. Increasingly, the moment that determines whether a customer chooses you happens inside a model's response — before a human ever sees your homepage.

This agentic audience is already here: by our count, fewer than half of all HTML page requests now come from a human. Not all of those machines are agents acting for a person, but that share is growing fast, and answer engines, shopping assistants, and research tools will shape which businesses are found and recommended. Discoverability used to mean ranking on a results page. Now it means being found, read, and confidently recommended by the agents that guide your customers.

The old metrics, human clicks and page views, no longer paint the full picture. We spent time talking to site owners who were staring at access logs full of AI bots, completely blind to whether those bots were capable of using their site or recommending Continue reading

Making AI search smarter

Search drives most experiences on the web. It's how we get things done, and how nearly everything on the web gets found — the creators, the merchants, the answer to whatever you just typed into a box. For nearly 30 years, that discovery journey ran on a simple bargain: let a search engine crawl your content, and it sends you visitors. You turned those visitors into a business — through ads, subscriptions, or just the audience itself. Being discoverable and getting paid were the same thing. A year ago, on the first Content Independence Day, we drew a line to defend that bargain in the AI era. But a line in the sand was only a first step. Since then, the prevalence of AI search in consumers’ lives has only accelerated as more than 50% of traffic online is non-human. The threat is no longer a handful of training crawlers you can block; it's search itself being rebuilt around AI answers.

Today's answer engines read your page and hand the user a summary, so the visit — and the revenue that depended on it — isn’t needed. We see it firsthand, and independent research backs it up: a 2025 Continue reading