AI search: how ChatGPT, Google AI and others find you

How the AI services find sources#

When someone asks an AI service “which electrician in Luleå is the best?”, the service often searches the web and summarises what it finds. Sites that can be read, are clear and are mentioned by others have a better chance of becoming a source. Each service has its own way, though, and none of them publishes a rule for exactly which sources are chosen.

Google's AI Overviews and AI Mode#

Google says the same rules apply as for ordinary search: a page that is indexed and may be shown with a snippet can also be shown as a link in AI Overviews and AI Mode. No special AI file or new markup is needed [1].

Traffic from AI Overviews is counted in the ordinary “Web” traffic in Search Console, so it cannot be seen separately there [1]. Anyone who does not want Google to use the content to train other AI models can use Google-Extended, without it affecting Google Search [2].

ChatGPT, Claude and Perplexity#

The services have different programs for different purposes, and robots.txt can control them one by one:

• OpenAI: OAI-SearchBot fetches pages for ChatGPT's search, GPTBot collects text for training, and ChatGPT-User visits a page when a user asks for it. Changes in robots.txt for the search can take about a day to take effect [3].
• Anthropic (Claude): Claude-SearchBot for search results, ClaudeBot for training and Claude-User when a user asks. They follow robots.txt [4].
• Perplexity: PerplexityBot for the search index and Perplexity-User for the user's own questions [5].
• Microsoft Copilot and Bing: build on Bing's index. Bing now shows a report on how often a site is cited in AI answers, in preview and without covering everything [6].

We check whether the search programs (the ones that decide whether you can appear in the answers) are let in, and show it in the analysis. If you want to shut out the training programs, that can be done without the search being affected.

This is what we do#

• Facts that machines read directly: company name, address and phone as structured data [7].
• An llms.txt file that summarises the site. It is a proposed standard [8]. Google does not need it [1], but it costs nothing and can help other services.
• Clear answers to the questions customers actually ask, on pages that can be read without logging in or JavaScript.

How we measure AI visibility, and why the figure is a sample#

We ask the AI services a number of questions about your industry and town and note whether your company is mentioned. The result shows which questions, which service and which date. AI answers change from one time to the next and from person to person, so the figure is a sample, not an answer key. Without the questions, the service and the time, an AI figure cannot be compared, and that is why we always show all three.