Short answer: On 30 October 2026 Google announced plans to support a shared robots.txt control for opting out of AI training that works across AI providers in one step, aiming to start in 2027. If a site opts out with this control and does not mention Google-Extended, Google will treat it as blocking Google-Extended too. Nothing needs changing yet, but businesses that want AI to recommend them should be clear that opting out of AI training and disappearing from AI answers are different things. Daaistudio sums up what changed and how we are adapting client work.
What changed
Writing on the Google Search Central blog on 30 October 2026, John Mueller of Google said "we're announcing our plans to support a shared AI training control in robots.txt." (Google Search Central) The key points:
- One control, many providers: a policy token in robots.txt lets a site say its content may not be used for AI training by any participating provider, without blocking bots one by one.
- Based on an IETF draft: it builds on the AIPREF working group's work, see draft-kelley-aipref-training-preference-signal. The announcement gives no syntax example yet.
- Effect on Google-Extended: Google-Extended, Google's AI training control since 2023, keeps working. If a site opts out with the new token and does not mention Google-Extended, Google plans to treat that as disallowing Google-Extended.
- Gemini: Google states that sites which specifically allow Google-Extended are eligible to appear in Gemini Apps responses and receive clicks via grounding.
- When: Google aims to support it with other AI providers in 2027 and will share progress.
The announcement says nothing about Google Search, AI Overviews or AI Mode, so we do not draw conclusions beyond what Google wrote.
What it means for Thai businesses
- No rush yet: this is a plan for 2027 and the standard is still a draft. If anyone sells you an "AI block" based on it today, ask which document they are following.
- Opting out of training is not leaving Search, but read the detail: Google's documentation states Google-Extended does not affect Google Search and is not a ranking signal. For Gemini Apps, though, allowing Google-Extended is what makes a site eligible to be cited with links. Businesses that want Gemini to recommend them should not switch on a blanket opt-out without thinking.
- Most business sites want AI to know them: restaurants, clinics, hotels and factories gain when AI answers customers with correct facts. News publishers or paid-content creators may weigh protecting their content more.
- Search bots are still separate per provider: for example, OpenAI separates OAI-SearchBot (which shows sites in ChatGPT search) from GPTBot (training). A training setting should never block a search bot by accident.
How we are adapting our client work
This news does not require changes to client sites today, but it adds a few points to work we already do.
- robots.txt checked in every site analysis: in the free analysis plan and the month-1 setup we look at which bots robots.txt blocks, because "block AI" plugins can block more than the owner meant. We have added this new token to the list to check once Google supports it.
- Two questions for the owner to decide: we separate them clearly: do you want AI to train on your content, and do you want AI to find and recommend you. The decision belongs to the site owner; we explain the effect of each choice from the official documents.
- Monthly AI tests, as before: on the SEO GEO plan we ask each AI tool real customer questions every month and keep screenshots in the report. If a robots.txt setting changes, these tests show the effect.
- Waiting for the real documentation: we will not add this token to client sites until Google publishes the supported format.
An example from our work: Brasserie 9, a French restaurant on Sathorn Soi 6 where we run monthly SEO and GEO. ChatGPT search picks it first for a question about French Sunday brunch in Bangkok. An AI finding and citing a site like this comes from a search bot that can read the site, a different switch from opting out of AI training.
ChatGPT search picks Brasserie 9 first for a question about French Sunday brunch in Bangkok (screenshot from the client report, September 2026). AI answers can differ by time and by who asks.
We hold a 5.0 rating on Google and Fastwork, from 18 Fastwork jobs with 8 repeat clients, and Al Saray and Brasserie 9 renewed for 6 months after their first 3.
What to do now: a short checklist
- Open your site's robots.txt (yourdomain/robots.txt) and see whether Google-Extended, GPTBot, OAI-SearchBot or PerplexityBot are blocked, and who added the rules
- If you use a "block AI" plugin, check whether it blocks only training bots or search bots too
- Decide clearly whether the business values AI recommendations or content protection more
- Do not add the new token until Google publishes the supported format
- Ask AI tools the questions real customers ask, and keep screenshots to compare before and after any robots.txt change
Not sure which AI bots your robots.txt blocks?
Send us your site. We check robots.txt, test the questions customers are likely to ask AI, and show you what to fix first, free.
Read where each AI tool finds its sources in What AI search is, and the bigger picture in SEO, AEO, AIO and GEO explained. No one can guarantee Google rankings or AI answers; we show only real, measurable results.
Sources
- Google Search Central Blog: Supporting a shared AI training control for websites (30 Oct 2026)
- IETF Datatracker: draft-kelley-aipref-training-preference-signal
- Google Crawling Infrastructure: Google's common crawlers, Google-Extended
- OpenAI: Overview of OpenAI crawlers
- Google Search Central: AI features and your website