Most AI search measurement I see is one person typing a question into ChatGPT, seeing whether the brand comes up, and taking a screenshot. That tells you what one model said to one account in one place at one moment. Run it again and the answer may change. The first-party reports have arrived, but each covers one provider and one kind of event, so a team can easily read a rise in one number as a win everywhere. A method that holds up needs a small, fixed prompt set built from the services you sell, sampled repeatedly, with citations recorded and joined to Search Console, Bing Webmaster Tools and analytics.
The mechanics behind why answers vary, from query fan-out to the crawlers involved, are covered in the AI search guide.
What each source can and cannot tell you
Start by being clear about what each source measures. They are not interchangeable, and most reporting mistakes come from treating them as if they were.
| Source | What it measures | What it misses |
|---|---|---|
| Your own prompt sampling | Whether you are named or cited for prompts you chose, in any assistant you test | Only covers the prompts you wrote. Real users ask things you did not think of. |
| Search Console, Generative AI performance report | Impressions of links to your site in AI Overviews and AI Mode, by page, country, date and device1 | Clicks and queries are not listed as metrics. ChatGPT, Copilot, Claude and Perplexity are outside it. |
| Search Console, main Performance report | Clicks from AI features, counted under the Web search type alongside ordinary results2 | AI feature clicks are mixed in with classic results, so you cannot isolate them there. |
| Bing Webmaster Tools, AI Performance report | Citations in Copilot, Bing AI summaries and some partner integrations, cited pages, and a sample of grounding queries3 | Clicks and placement within the answer3 |
| Analytics referrals | Visits that arrive with an assistant as the referrer or a tagged source | AI Overview and AI Mode clicks look like Google organic. Visits that lose the referrer, and buyers who come back later by another route. |
Build a small prompt set from your services and seed keywords
Your prompt set is the only part of this you design, so it decides how useful everything else is. Start from the type of business and what it wants from the assistants. A plumber wants to be named when someone nearby needs a repair today, and a SaaS company wants to be on the shortlist when a buyer compares tools. A shop wants its products to come up for the categories it sells. The prompts worth tracking follow from that, and from the searches you already know bring in customers.
Do not track hundreds of prompts
It is tempting to track every way a customer might phrase a question. That gets expensive quickly and tells you very little more. By hand, every prompt is multiplied by the assistants and the runs: 200 prompts in four assistants at five runs each is 4,000 answers a month. Paid tools charge by the prompt. OtterlyAI's monthly plans in October 2026 ran from $29 for 15 prompts to $489 for 400, with extra prompts at $99 per hundred on the larger plans,4 and Peec AI's plans are set at 50, 150 or 350 prompts, with the monthly number of answers worked out as prompts times models times tracking frequency.5
A large set is also less necessary than it looks, because of the way assistants search. Google describes query fan-out as a set of related searches the model generates at the same time to request more information. Its example turns “how to fix a lawn that's full of weeds” into searches for the best herbicides for lawns, removing weeds without chemicals, and preventing weeds in a lawn.6 Nectiv ran more than 8,500 commercial prompts through ChatGPT and recorded a search on 31% of them, with an average of 2.17 searches. The searches averaged 5.48 words, which the study's author describes as slightly longer than the keywords most SEOs track.7
In Google's example every search stays on the subject of the question, the way the keywords you already track do. So differently worded prompts about the same service tend to run similar searches, and one prompt written close to a seed keyword stands in for many ways of asking it. SparkToro's 2026 study points the same way: 142 prompts that volunteers wrote for one task were barely alike in wording, yet the leading headphone brands appeared in 55% to 77% of the 994 answers.8 What you want is a small set, organised by service and seed keyword.
Organise the set by service group and seed keyword
Build the set the way the business is organised. List the main service groups, the ones that bring in most of the revenue or that you most want to grow. For a shop these are the main product categories, and for a SaaS product the main jobs it is bought for, such as rota planning or payroll export. Under each group, take the two or three seed keywords you already track for SEO. Then write one or two prompts for each seed keyword, staying close to it: one that asks for the service with the place or qualifier customers add, and, where it fits, one that adds the customer's situation or asks for options to be compared.
Add two or three brand checks that name the business, and keep them apart when you report. For most businesses the whole set comes to between 10 and 30 prompts. That covers every service that matters, is small enough to run by hand, and stays cheap if you move to a tool later. A business with many service groups or several markets will need more, but add them a group at a time, once the routine runs smoothly.
Laid out as a map, the set shows its own gaps. Draw each service group as a slice and each prompt as a point, and a service nobody is measuring stands out at once. SearchOps, the platform I built, organises prompts by service in the same spirit: it groups a business's keywords into the services it sells and writes prompts per service, so fifty ways of searching for one service are asked about once.9
A prompt map for a local vet, by service group
Vaccinations3 tracked · 1 suggested
Asks for the service
- “Where can I get puppy vaccinations in Bristol?”
- “Which vets in Bristol do cat vaccinations?”
Adds a situation
- “My puppy is eight weeks old and needs her first jabs. Which vets near Clifton can see us this week?”
Compares options
- “Compare vaccination prices at vets in Bristol”
The same structure works for any business. The table shows one service group and seed keyword for three more, with the prompts each would track for it. The tutorial on getting recommended by AI assistants works through a whole set for a local business.
| Business | Service group and seed keyword | Prompts tracked |
|---|---|---|
| SaaS: rota and scheduling software for care homes | Staff scheduling: “care home rota software” | “What rota software works well for a 60-bed care home?” and “Compare three care home rota tools on price per staff member and how they handle agency staff.” |
| Ecommerce: outdoor clothing retailer | Insulated jackets: “synthetic insulated jacket” | “Which synthetic insulated jackets under £200 are good for wet weather in the UK?” and “Compare synthetic insulated jackets under £200 on weight, packed size and warmth when wet.” |
| B2B: industrial cleaning contractor | Food factory cleaning: “food factory cleaning west midlands” | “Which contractors do overnight food factory deep cleans in the West Midlands?” and “Compare food factory cleaning contractors in the West Midlands with BRCGS experience, and say which publish prices.” |
Take the keywords from data, and use AI only to brainstorm
The seed keywords, and the wording you add to them, should come from data you already hold:
- The keywords you already track for SEO, with their search volumes and the pages that rank for them.
- Search Console queries for each page, especially the long ones, which show the places, situations and qualifiers people add.
- The grounding queries in Bing Webmaster Tools' AI Performance report: the key phrases the AI used when it retrieved content that was referenced in its answers. Microsoft says the data is a sample.3
- Search terms from paid search campaigns, if you run them.
- The searches that find you on Maps, for anything local.
- The searches an assistant ran while answering. ChatGPT reports the searches it ran, so a tracker can collect them.10
- Real enquiries, sales calls and reviews, which give the customer's own words for the situation. Remove names, phone numbers, email addresses and street addresses before you paste them into any tool.
ChatGPT, Claude or Gemini can help you brainstorm wording around a seed keyword, and that is a reasonable use of them. Treat what they return as suggestions. A list written by an assistant is not data: it has no idea how often anything is asked, it comes out different every time you ask, and it can drift away from what you actually sell. Keep the suggestions that match your own data, and make sure every prompt you track traces back to a service group and a seed keyword you have evidence for.
These are my working rules for the set. Nobody has published a standard for this yet.
- Keep most prompts unbranded. A prompt that names you measures whether the assistant can describe you, which is worth knowing, but it says nothing about whether you are recommended. Keep the brand checks separate in the reporting.
- Stay close to the seed keyword. Write the way people type to an assistant, in full sentences, and add only the constraints customers really add, such as a place, a budget, a size or an integration, taken from your own data.
- Avoid leading prompts. “Is our brand the best option for X?” will usually get you named.
- Include location where it matters. For anything local, write the place into every unbranded prompt, and if you cover several towns, use your two or three most important ones. For an online service or a shop, name the country.
- Fix the set. Add prompts when you find a seed keyword you were missing, but never edit an existing prompt in place, or the trend line breaks.
Sample repeatedly and control the conditions
An assistant does not return a fixed result for a prompt. AI Mode and AI Overviews may use different models and techniques, so the responses and links they show will vary.2 OpenAI says ChatGPT may infer a general location from the IP address, and may use saved memories when it rewrites a prompt into search queries.11 So the same prompt can produce different shortlists for two colleagues, or for one person on two days.
The answer is to treat every result as a sample and control what you can. This is the routine I would follow.
Personalisation: measure the customer asking cold
Assistants personalise. A signed-in person's history, memories and location shape what they see, and nobody outside can reproduce that answer exactly.10 Chasing it is a trap. The case worth measuring is the customer asking cold, with nothing to personalise on, because that is the answer you can repeat and the one a new customer most likely gets.
- Put the personalisation into the prompt, where it is visible and repeatable. Write the town or city into every local prompt, as SearchOps does,9 and write the buyer's constraints into the rest: budget, team size, the integration they need.
- Treat a persona as a set of constraints, not an account. A 60-bed care home manager and a two-site franchise owner write different prompts, so where both matter, give each its own prompt under the same seed keyword and count it in the set.
- Report each engine separately. SearchOps runs every prompt on ChatGPT and Gemini and never blends the two,10 because they draw on different sources and an average of the two describes neither.
Record citations, not just mentions
Being named and being cited are different events. An assistant can recommend you while citing a review site, a directory or a competitor's comparison page as its source. It can also cite your page for a fact without recommending you. So record every answer as structured data, the same fields every time, and keep the full text so you can re-read it later.
One answer, recorded as data
PromptWhich vets near Clifton, Bristol are good with nervous dogs? I need one open on Saturdays.
Here are three options near Clifton that are known for handling anxious dogs and open on Saturdays. Harbourside Animal Clinic offers longer, quieter appointments and has a Saturday morning surgery. Elmbrook Vets is mentioned in several reviews for patient handling of nervous pets. Downs Veterinary Centre also opens on Saturdays. Sources: a local pet directory, a review site.
- You
- named, position 2 of 3, no stated reason
- Businesses named
- Harbourside Animal Clinic, Elmbrook Vets, Downs Veterinary Centre
- Named with a reason
- Harbourside: yes · Elmbrook: no · Downs: no
- Cited sources, by type
- directory (cites rivals), review site
- Searches the engine ran
- vets clifton bristol nervous dogs; vets open saturday bristol
- Outcome
- Mentioned (shortlisted would need a reason)
- Since last check
- new: absent in the previous check
- Date, assistant, model or mode, logged-in state, location and run number.
- Whether you were named, and whether the answer gave a reason for naming you. Being named with a reason is a stronger result than a bare listing.
- Every cited URL, marked as yours, a competitor's or a third party's, and the kind of source: directory, review site, publication, official body.
- The searches the assistant ran, where it discloses them.
- How the business was matched. For local businesses, match on website, phone number or name and address, not the name alone, which is often ambiguous.
- An outcome label, such as shortlisted, mentioned, cited only or absent, and how it compares with the previous check: new, consistent or lost.
- Which competitors were named.
- Any wrong fact about you: an old price, a discontinued product, a service you do not offer. For prompts that name you, also record the verdict: favourable, mixed, neutral or critical. Grounding does not prevent this: Microsoft's transparency note for Copilot warns that a response might misrepresent the content it was grounded in.12
The third-party citations are where the work is. If a care home software prompt keeps citing the same independent comparison article, the questions become whether you are in it and whether it describes you accurately. If an outdoor retailer's jacket prompts cite a rival's size guide, the gap may be a missing size guide on your own site. The mentions tell you how you are doing. The citations tell you what to do next. The most useful single view is a list of sources that cite your competitors but never you, grouped by type.9
An assistant writes a new answer every time, in a new order, so a single "you are number 2" is not a number anyone can defend.9 The figures that hold up are pooled across every response in a period:
| Metric | What it means | How to calculate it |
|---|---|---|
| Visibility rate | How often the answers include you | Responses that name you, divided by all responses, pooled over every prompt and check in the period |
| Share of voice | Your share of all the businesses named | Times you were named, divided by all business mentions in those responses |
| Named with reasons | How often the assistant explains why it chose you | Responses that give a reason for naming you, divided by responses that name you |
| Consistency | Whether you are named reliably or occasionally | The share of checks in which a prompt names you, per prompt |
| Engine agreement | Whether ChatGPT and Gemini agree about you | Prompts where both name you, or neither does, divided by all prompts |
Pool the counts; never average averages. Five prompts at 100% and one at 0% is not 83% if the zero prompt ran four times as often. Read the trend over 30 and 90 days, and compare the first half of a window with the second to see the direction, which is how SearchOps reports movement.9 A rise inside one week is usually noise.
Join it up with Search Console, Bing and analytics
Search Console
The Generative AI performance report counts impressions, defined as how many times links to your site were shown in a generative AI feature on Google Search, and groups them by the final URL after redirects.1 Standard Performance report limits such as the 1,000 row limit apply, and the newest data is preliminary.1 Compare its top pages with the URLs your prompt set sees cited in AI Mode. Where they disagree, your set is probably missing a service group or seed keyword that real users search for.
When you read clicks in the main Performance report, remember the counting rules. All links in an AI Overview share its single position, a click on an external link in an AI Overview or AI Mode counts as a click, and a follow-up question is treated as a new query.13 Position figures for pages that appear mostly in AI Overviews will look better than the experience feels. Do not read AI Overview impressions as a stand-in for clicks either. Pew Research Center tracked the Google searches of 900 US adults in March 2025 and found that people clicked a link inside an AI summary in 1% of visits to pages that showed one, and clicked a traditional result in 8% of those visits, against 15% on pages without a summary.14 Seer Interactive, an SEO agency, found that across 3,119 informational queries for 42 client organisations, brands cited in an AI Overview had a 35% higher organic click-through rate than brands that were not, which is a correlation and supports recording citations separately from mentions.15
Bing Webmaster Tools
The AI Performance report shows total citations, average cited pages, page-level citation activity and grounding queries, the phrases the AI used when it retrieved content it went on to cite.3 Microsoft says the grounding query data is a sample.3 Use those queries to check your seed keywords and the wording of your prompts. A B2B contractor might find grounding queries about accreditation or site induction rules that nobody on the team would have written as a prompt.
Analytics
OpenAI's publisher FAQ said that ChatGPT adds utm_source=chatgpt.com to referral URLs, so those visits can be tracked in analytics.16 OpenAI took that page down in October 2026 and no current OpenAI page repeats the statement, so check your own landing page reports for the parameter and segment on the chatgpt.com referrer as well. For other assistants, check your referral report for their domains and confirm what actually arrives before you write any rules. In GA4, a custom channel group can collect these sources into one channel using conditions such as “source matches regex”. Traffic goes to the first channel it matches, so place the AI channel above Referral, and custom groups apply to reports retroactively.17
Treat this channel as a floor. A SaaS buyer who sees you in a comparison table and comes back a week later through a branded search shows up as organic or direct, with no trace of the assistant. For that reason I would also watch branded search impressions in Search Console and add an “assistant” option to “how did you hear about us” fields on demo and quote forms.
Where a tracking tool fits, and what to ask of one
A spreadsheet works until the prompt count, the engines and the schedule multiply. At that point a tool earns its keep, provided it shows its method. Most price by the prompt, so a small set keeps that cost down as well. Google says no third-party tool has access to its internal ranking or AI systems,6 so every tracker is sampling, as you are. Ask any vendor to show you the prompts, the run counts, the locations and how they count.
SearchOps automates the method on this page for local businesses. It runs every tracked prompt on ChatGPT and Gemini on a weekly, fortnightly or monthly schedule and reports the engines separately.9 It stores the full response, the businesses named, the searches ChatGPT ran and the citation sources, matches your business by website, phone number or name and address, and compares each check with the last.9 Its reports group cited sources by type and flag the ones that cite rivals but never you, score the verdict on branded prompts,9 and show AI Overview and AI Mode citations beside the assistant results.10 You can see the details on the ChatGPT Visibility page.
Outside local, the same method still applies. A SaaS team or an ecommerce manager can run it in a spreadsheet, or ask any tool they consider to show the same evidence.
Mistakes that make the numbers lie
- Reporting a single run as a position. One answer is an anecdote.
- Tracking hundreds of prompts, or a list an assistant wrote. Every extra prompt multiplies the cost, and an AI-written list says nothing about what customers ask.
- Testing from your own logged-in account, where memory and history pull the answer towards things you have asked before.
- Adding branded prompts to the headline figure, which inflates it with prompts you were always going to win.
- Comparing Search Console impressions with Bing citations as if they measured the same event. One counts links shown; the other counts sources cited.
- Reading Bing's grounding queries as a complete list. Microsoft describes them as a sample.3
- Buying a single visibility score without knowing the prompts and sampling behind it. Every tool is sampling, as you are. Ask to see the prompts.
- Changing the prompt set and the site in the same month, then crediting the site for the difference.
None of this needs special software to start. A list of service groups and seed keywords, 10 to 30 fixed prompts, a run log and a monthly summary will tell you more than a dashboard built on prompts you cannot see. Once the routine works, automate the running and keep the method.
Common questions
01How many prompts should I track to measure AI visibility?
02Can I use ChatGPT to write my prompt list?
03Can Search Console show which queries trigger AI Overviews for my site?
04How do I track traffic from ChatGPT in Google Analytics?
utm_source=chatgpt.com to referral links.16 Check that it still arrives in your own data. You can group that source, with other assistant referrers, into a custom channel group in GA4 and place it above Referral so it is matched first.1705How many times should I run each prompt?
06Does the Bing AI Performance report show clicks?
07Can I track what ChatGPT shows a signed-in customer?
08Are AI visibility tracking tools accurate?
Sources
- Platform docs13
- Industry study4
- 01Generative AI performance report (Search)Search Console HelpPlatform docs
- 02AI features and your websiteGoogle Search CentralPlatform docs
- 03Introducing AI Performance in Bing Webmaster Tools Public PreviewBing Webmaster Blog (February 2026)Platform docs
- 04OtterlyAI PricingOtterlyAI (October 2026)Platform docs
- 05Pricing for Peec AIPeec AI (October 2026)Platform docs
- 06Optimizing your website for generative AI features on Google SearchGoogle Search CentralPlatform docs
- 07New Data Study: What Queries Is ChatGPT Using Behind The Scenes?Nectiv (Chris Long, October 2025)Industry study
- 08NEW Research: AIs are highly inconsistent when recommending brands or products; marketers should take care when tracking AI visibilitySparkToro, Rand Fishkin with Gumshoe.ai (January 2026)Industry study
- 09ChatGPT VisibilitySearchOps (the author's company)Platform docs
- 10Gemini VisibilitySearchOps (the author's company)Platform docs
- 11Searching the web with ChatGPTOpenAI Help CenterPlatform docs
- 12Transparency Note for Microsoft Copilot (for individuals)Microsoft SupportPlatform docs
- 13What are impressions, position, and clicks?Search Console HelpPlatform docs
- 14Google users are less likely to click on links when an AI summary appears in the resultsPew Research Center (July 2025)Industry study
- 15AIO Impact on Google CTR: September 2025 UpdateSeer Interactive (November 2025)Industry study
- 16Publishers and Developers - FAQ (archived 4 October 2026)OpenAI Help Center, via the Internet ArchivePlatform docs
- 17Custom channel groupsAnalytics HelpPlatform docs
Written by Mani Bharij, SEO & AI Search Consultant in London. More on this subject in the AI search guide.