TL;DR Summary:
New Clarity Metric: Microsoft Clarity now shows an AI Scrape-to-Referral Ratio in Bot Analytics, letting site owners compare how much AI bots take versus how much traffic they send back.Big Traffic Gap: Microsoft’s example showed about 6,000 scrapes for every referral, while Cloudflare previously found Anthropic’s crawler at nearly 71,000 requests per visitor.Why It Matters: The metric gives marketers a more direct way to judge AI traffic value, but low referrals can be misleading if bots are blocked or platforms hide referrer data.Use It Carefully: The number is most useful when paired with crawler-access checks and session data, since it does not fully capture brand visibility inside AI answers.How much does an AI crawler take from your site compared to what it sends back? Microsoft Clarity now answers that question with a new AI Scrape-to-Referral Ratio, live in its Bot Analytics dashboard as of August 19, 2026. The timing matters because publishers have spent months arguing that AI companies take content without giving traffic in return, and now there is a number attached to the claim.
What the AI Scrape-to-Referral Ratio Shows on Your Site
The new card sits inside Clarity’s Bot Analytics dashboard. It pairs bot-crawl data with referral analytics so you see both sides at once. Microsoft’s own example put the average at roughly 6,000 scrapes for every one referral. The card also ranks AI operators by how much traffic they send versus how much they take, and it links to filtered session recordings. You can watch what an AI-referred visitor actually does after landing on your page. Clarity limits its math to correctly mapped domains, so a coverage mismatch does not throw off your ratio. Before treating any of these numbers as final, it’s worth confirming that AI crawlers can even reach your content in the first place—a low referral count could simply mean a bot never got past your site’s technical barriers. ClickRank is built for exactly that check, testing whether crawlers from ChatGPT, Claude, Gemini, and Perplexity can actually access your pages before you start interpreting scrape-to-referral figures.
How the Cloudflare Findings Set Up This Metric
Cloudflare published a report in July 2025 that found Anthropic’s crawler made nearly 71,000 requests for every one visitor it referred over a single week in June. Cloudflare now blocks AI crawlers by default and runs a pay-per-crawl marketplace, where sites can charge bots for access. Cloudflare’s CEO, Matthew Prince, said AI crawlers “have been scraping content without limits,” and said his goal is to put “the power back in the hands of creators.” Cloudflare cautioned that its own number can overstate the problem, since many AI apps send no referrer data at all. The AI Scrape-to-Referral Ratio in Clarity gives you a version of that same comparison, but scoped to your own domain instead of Cloudflare’s full network.
Why This Ratio Changes What Marketers Track
Most AI visibility tools count how often a brand shows up in AI answers. The AI Scrape-to-Referral Ratio instead tracks what those appearances send back in actual site traffic. That is a different question, and it matters more as AI search grows. Semrush projects that AI search visitors will overtake traditional search visitors by 2028. If that holds, the gap between being mentioned and getting a visit will only grow more important to measure.
The Two Blind Spots in This Number
A high scrape count with few referrals does not automatically mean your AI visibility is failing. An AI answer can build awareness for your brand without ever producing a click. Furthermore, the referral side of the ratio likely undercounts real traffic, since many AI platforms send no referrer and those visits show up as direct traffic instead. Clarity measures your single domain and connects the numbers to session recordings. Cloudflare measures the same gap across its entire network. Neither tool tells you how you compare to a competitor inside the AI answers themselves.
Watch your own AI Scrape-to-Referral Ratio in Clarity, but do not treat a low referral count alone as proof that AI visibility has failed you. Check whether the AI platforms sending you traffic are passing a referrer at all, since that gap can hide real visits. Before you dig too deep into your scrape-to-referral ratio, it’s worth confirming AI crawlers can actually reach your pages in the first place—ClickRank’s crawler-access check tells you whether bots from ChatGPT, Claude, Gemini, and Perplexity are technically blocked from your content before you draw conclusions about low referral numbers. Pair this number with what you know about your brand’s presence inside AI answers before you draw conclusions.
