Google-Extended & robots.txt
Control Gemini AI Training
Block or allow Google's AI training crawler — without affecting your Google Search ranking.
What is Google-Extended?
Google-Extended is a crawler token introduced by Google in September 2023. It controls whether your content is used for training Google's AI products, including Gemini and AI Overviews in Search. It is separate from Googlebot — blocking Google-Extended does not affect your search ranking.
This separation is deliberate. Google created Google-Extended after publishers demanded a way to opt out of AI training without losing search visibility. You reference it in robots.txt as Google-Extended.
Google-Extended vs Googlebot
Googlebot
Search indexing. Affects your ranking. Never block this unless you want to disappear from Google.
Google-Extended
AI training only. No ranking impact. Safe to block if you want to opt out of Gemini training.
robots.txt Syntax for Google-Extended
Copy-paste these examples into your robots.txt file.
Allow Google-Extended (AI training opt-in)
# Allow Google AI training
User-agent: Google-Extended
Allow: /
Allows Google to use your content for Gemini and AI Overviews. Your content may be cited in AI-generated answers.
Block Google-Extended (AI training opt-out)
# Block Google AI training
User-agent: Google-Extended
Disallow: /
# Keep Googlebot allowed for search
User-agent: Googlebot
Allow: /
Blocks AI training while keeping your site in Google Search. This is what many major publishers use.
Block All AI Crawlers (Google + Others)
# Block AI training from all major providers
User-agent: Google-Extended
Disallow: /
User-agent: GPTBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: CCBot
Disallow: /
# Keep search engines allowed
User-agent: Googlebot
Allow: /
See our full list of 14 AI crawlers to block them all.
When to Allow vs Block Google-Extended
Allow Google-Extended When...
- ✓You want to appear in Google AI Overviews
- ✓You want Gemini to know about your brand/product
- ✓You publish public knowledge or documentation
- ✓You want maximum AI visibility across platforms
Block Google-Extended When...
- ✗You publish premium or paywalled content
- ✗You don't want Google training AI on your work
- ✗You have licensing or copyright concerns
- ✗You want search traffic but not AI summarization
Frequently Asked Questions
Does blocking Google-Extended affect my Google Search ranking?
No. Google-Extended only controls whether your content is used for training Gemini and other Google AI products. Your Google Search ranking is determined by Googlebot, which is a separate crawler. Blocking Google-Extended has zero impact on your position in Google Search results.
What's the difference between Googlebot and Google-Extended?
Googlebot crawls your site for Google Search indexing — it determines whether your pages appear in search results. Google-Extended crawls content specifically for training Google's AI models like Gemini. They are independent: you can block Google-Extended while keeping Googlebot fully allowed.
What does Google-Extended actually do with my content?
Google uses content collected by Google-Extended to improve Gemini (their AI model), Google AI Overviews in search, and other AI-powered features. If you allow it, your content may influence how Gemini responds to questions. If you block it, your content is excluded from future AI training data.
Should I block or allow Google-Extended?
If you want your content to influence Google's AI products (Gemini, AI Overviews) and potentially be cited, allow it. If you want to keep your content out of AI training data while maintaining your search ranking, block it. Many publishers block Google-Extended while keeping Googlebot allowed — it's a common and supported configuration.
When did Google introduce Google-Extended?
Google launched Google-Extended in September 2023 specifically to give publishers control over AI training data. Before this, there was no way to opt out of Google's AI training while keeping your site in search results. It was a direct response to publisher concerns about AI using their content.
Related robots.txt Guides
- GPTBot & robots.txt — OpenAI / ChatGPT
- ClaudeBot & robots.txt — Anthropic / Claude
- PerplexityBot & robots.txt — Perplexity AI
- CCBot & robots.txt — Common Crawl
- Check all 14 AI crawlers at once
Check Your Google-Extended Configuration
See if Google-Extended and other AI crawlers can access your site. Full report on all 14 AI bots.
No credit card required