Anthropic Bot Crawlers: Ranking & Little Contribution
- Google is increasingly answering search queries directly on its search results page (using AI Overviews and AI mode) and reducing traffic sent to external websites.
- * Crawl-to-Refer ratio: cloudflare data shows a meaningful increase in Google's crawl-to-refer ratio (how often Google visits websites vs.
- In essence, the article suggests Google is relying more on providing answers directly within search, potentially impacting the traffic websites receive from Google search.
Here’s a summary of the Business Insider article:
Google is increasingly answering search queries directly on its search results page (using AI Overviews and AI mode) and reducing traffic sent to external websites.
key Data Points:
* Crawl-to-Refer ratio: cloudflare data shows a meaningful increase in Google’s crawl-to-refer ratio (how often Google visits websites vs. how frequently enough it sends traffic to them).
* January: 3.3 to 1
* April: 18 to 1
* july: 9 to 1
* Google’s Response: Google claims it still sends traffic to the web and values the health of the web ecosystem.
* Future Tracking: Business Insider plans to continue monitoring this data to see how Google’s behavior changes over time.
In essence, the article suggests Google is relying more on providing answers directly within search, potentially impacting the traffic websites receive from Google search.
The article also includes a plug for Business Insider’s Tech Memo newsletter and contact facts for the author.