AI · 07

Getting found by search and AI answers (SEO & GEO)

How Google, Bing and AI assistants find your pages — and what you can, and can't, do to speed that up. SEO is search engine optimisation; GEO, generative engine optimisation, is the same for AI answers.

Last reviewed 15 October 2026. Search engines change their rules often — the links at the bottom are the sources to check.

The one idea

Nothing gets cited before it is found. Search engines run the same chain: discover → crawl → index → rank or cite. You can make the first steps easy and fast. You can't force the last one.

AI answers that cite sources mostly sit on top of those same search indexes, so good SEO is most of good GEO. Two exceptions: an assistant may fetch a page live when someone asks about it, and it may answer from what it learned in training, with no lookup at all.

The chain

Discover
The engine learns a page exists — from links to it, or from your sitemap (a list of your pages with the date each one last changed).
Crawl
Its crawler fetches the page. Your robots.txt asks crawlers to stay out of certain areas — a request about crawling, not a way to hide a page (see the myths below).
Index
The page is read, understood and stored. A canonical tag tells it which address is the real one when several show the same text.
Rank or cite
When someone searches or asks, the engine picks pages on relevance, quality and trust. This step is theirs, not yours.

Telling an engine "this page changed"

Google: it pulls
Google decides when to come back. It reads your sitemap and uses the change dates — but only if they are consistently accurate. It ignores the "priority" and "change frequency" fields, and retired its sitemap "ping" in 2023. For a few key pages you can ask by hand in Search Console (Request indexing).
Bing & co: you push
IndexNow lets a site send a notice the moment a page changes. One notice is shared by every engine that takes part: Bing, Yandex, Naver, Seznam, Yep and others. Google does not take part.
The Google exception
Google's Indexing API is a push channel, but only for job postings and livestream video pages. Google checks every submission for spam and can revoke access when the API is abused.
Either way
A sitemap or a notice is a hint, not an order. It gets a page looked at sooner; it doesn't guarantee it is indexed.

Where AI answers get their pages

Google AI Overviews & AI Mode
Google's own search index. Google says no extra files, markup or special optimisation are needed: a page that can show in Search with a snippet is eligible.
Microsoft Copilot
Turns the question into a short query for Bing search and answers from those results, so being in Bing's index is what counts.
ChatGPT search
OpenAI's own search crawler (OAI-SearchBot), plus search partners, Microsoft among them.
Other assistants
Perplexity and Anthropic (Claude), among others, publish the names of their own crawlers. Check that your robots.txt lets the ones you want in.

Search is not training. Most AI companies use one crawler to show your pages in answers and another to collect text for training models. robots.txt can treat them differently: blocking OpenAI's training crawler (GPTBot) doesn't remove you from ChatGPT search, and Google's training control (Google-Extended) has no effect on Google Search. It also won't keep you out of AI Overviews: they are part of Search, where the levers are nosnippet and noindex.

What actually helps

  • Let the right crawlers in — check robots.txt and any firewall rules that might block them.
  • One topic, one address — its own title, description and canonical. A page that covers ten topics can only be found as one.
  • Honest dates — a sitemap date that moves only when the content really changes.
  • Push where you can — IndexNow for Bing and its partners; Request indexing in Search Console for the few Google pages that matter most.
  • Answers that stand alone — clear headings, the key fact in the first sentence, text rather than text-in-images.
  • Structured data that matches the page — it helps engines read the page, but only when it says what the visible page says.

How to check

Google
In Search Console, URL inspection shows whether a page is indexed, when Google last crawled it, and which address it treats as the real one.
Bing
Bing Webmaster Tools has its own URL inspection. Copilot and the other engines in Bing's feed see what Bing has.

Common myths

  • "An llms.txt file gets you into Google's AI answers." Google says no special files are needed for its AI features. Some AI tools may read one, so it is cheap to keep — just not a lever.
  • "Submitted means indexed." Submitting is a request to look, nothing more.
  • "Sitemap priority decides what Google crawls." Google ignores it.
  • "Blocking AI training hides you from AI search." Different crawlers, different switches.
  • "robots.txt keeps a page out of search." A blocked page can still be indexed if other sites link to it. To keep a page out, use noindex or a password; to keep its text out of snippets and AI Overviews, use nosnippet.
Go deeper (the sources)
Google: build a sitemap Google: sitemaps ping endpoint is going away (2023) Google: AI features and your website Google: Indexing API IndexNow FAQ OpenAI: crawlers Google: crawlers and Google-Extended Google: what robots.txt is (and isn't) for Microsoft: how web search works in Copilot Anthropic: crawlers Perplexity: crawlers
← More in AIAll Knowledge Base topics →