Does Blocking Google-Extended Remove Me from AI Overviews?
No. Blocking Google-Extended in robots.txt does not remove your pages from AI Overviews. Google-Extended is a separate crawler token that controls whether your content can be used to train and ground Google's generative models, such as Gemini and Vertex AI. AI Overviews eligibility is governed by Googlebot and standard snippet controls, not Google-Extended. Disallowing Google-Extended affects model training, not what appears in Search or its generative panels.
What Google-Extended is for
Google documents Google-Extended as a robots.txt user-agent token that publishers use to manage whether their content helps improve Gemini and Vertex AI generative APIs. Adding User-agent: Google-Extended with Disallow: / removes your site from that training and grounding use. Crucially, it does not change how Googlebot crawls or indexes you, does not alter rankings, and does not make you ineligible for AI Overviews.
Which control governs what
| Control | Governs | Affects AI Overviews? |
|---|---|---|
| Google-Extended (robots.txt) | Gemini / Vertex AI training & grounding | No |
| Googlebot (robots.txt) | Search crawling & indexing | Yes — blocking de-indexes you |
nosnippet / max-snippet | Snippet & preview usage | Yes — suppresses Overview inclusion |
data-nosnippet | Specific wrapped passages | Yes — for the wrapped text only |
Why the confusion is common
The names invite it: "Google-Extended" sounds like it covers Google's newer AI surfaces, and AI Overviews are one of those surfaces. But Google separated the levers deliberately. Training data collection (Google-Extended) is decoupled from the retrieval-and-synthesis that powers AI Overviews and AI Mode, which pull from the live Search index in real time. So a page you excluded from Gemini training can still be crawled by Googlebot and cited in an Overview.
What to do instead
If your goal is to stay out of AI Overviews, use snippet controls — but know they also kill featured snippets and previews; see can you opt out of AI Overviews. If your goal is the opposite — to be cited in Overviews — leave Googlebot fully allowed, write answer-first passages, and add original data, since the Aggarwal et al. GEO paper (KDD 2024) measured a 30-40% generative-visibility lift from citations and statistics. Then confirm reality with AI visibility monitoring rather than assuming a robots directive changed your Overview presence. The GEO optimization guide covers the crawler-access details.
Bottom line: Google-Extended is a training opt-out, not a Search or AI Overviews opt-out. To influence Overviews, use snippet controls and citation-worthy content.
Frequently asked questions
- What does Google-Extended actually control?
- Google-Extended is a robots.txt token that governs whether your content can be used to train and ground Google's generative models like Gemini and Vertex AI. It does not affect Googlebot, Search indexing, ranking, or AI Overviews eligibility, which are handled by separate crawlers and controls.
- How do I actually opt out of AI Overviews?
- Use snippet controls — nosnippet, max-snippet, or data-nosnippet. These suppress AI Overview inclusion but also remove featured snippets and text previews. There is no Google-Extended setting that removes you from AI Overviews only.
Keep exploring
See how AI engines talk about your brand — track mentions across ChatGPT, Perplexity, Claude, Gemini and 5 more. Start with Menra