A single favourable AI answer is a starting observation. To understand whether buyers are finding your business, keep the questions fixed, save the full answers and repeat the check. That gives you evidence to act on when a competitor keeps appearing before the next enquiry.
For an agency, the value is a clear client decision. Which questions reveal wrong facts? Which services have weak supporting pages? Which sources repeatedly name other providers? A small, transparent baseline is more useful than a score whose questions and denominator are hidden.
Why does ChatGPT give different answers?
Answers can vary with wording, context, location, product settings, search behaviour and time. Changing several of these together makes a comparison hard to interpret. Repeating the same prompt also does not make every answer a fully independent measurement of the market.
Keep the exact wording and record the product or model, time, market and session conditions. Report the observations you collected. “Named in two of six usable answers in this test” tells a client what happened. “Invisible to AI” is a much broader claim that the evidence does not establish.
Which questions should I test?
Use three groups because they represent different decisions. Brand questions check what an assistant says about a supplied name. Provider questions ask for a business that meets a need. Problem questions describe the situation that makes a person seek help.
For example: “What services does [business] offer?”, “Which plumbers serving [town] handle emergency leaks?” and “What should I ask before choosing someone to replace a water heater?” Label these as illustrative prompts unless they came from a saved customer enquiry or another verified source.
Start with a manageable set across your important services. Include costs, availability, alternatives, comparisons and trust. Use the language callers use. Keep brand-named questions outside any claim about discovery without a supplied brand. A business being recognised when named is different from being suggested spontaneously.
Test. save. repeat.: Fixed questions, Saved answers, Repeat checks.
Does ChatGPT know my location?
Make the intended town or service area explicit in the question. Record the location conditions you can control instead of assuming your office connection represents every buyer. Avoid “near me” for a multi-town comparison unless you can explain where the test was run.
For ChatGPT, OpenAI's current Temporary chat guidance offers a choice of Personalized or Unpersonalized before the first message. Temporary alone does not mean personalization is off. Use a consistent setting, or record a signed-out check where available. These steps reduce known context; they do not make results universal.
How many times should we ask?
For a first inspection, try a small fixed set more than once and on different days. Choose a volume your team can actually review. A handful of runs can expose an incorrect address or a useful competitor source, but it does not deliver a precise probability of being recommended.
Do not treat a missing AI feature, access error or interrupted answer as a successful answer omitting the business. Keep failed and unavailable checks visible. When comparing rounds, use the same denominator policy and show how many usable observations you have.
What should go in the worksheet?
Use one row per answer. Save the exact question, question group, platform or API model, date and time, service area, session conditions, full text, source URLs and any observed errors. Then record three separate findings: named, cited and recommended.
A mention is the business name appearing. A citation is a link or source reference. A recommendation is an expressed suggestion to consider the business. Preserve the exact wording that supports each label. If the classification is uncertain, keep that uncertainty visible rather than forcing a positive score.
Where a research system reuses a saved answer across comparisons, retain its original answer ID. Multiple target observations do not become multiple independent AI requests. Likewise, an API response should be labelled with its actual model; it is not automatically a test of the corresponding consumer chat product.
What should we fix when competitors appear?
Read the cited page. Does it explain a relevant service, an accurate local profile, a price factor or a decision the buyer faces? Verify your own equivalent information before planning a new page. Check access if your site cannot be read, using the website access guide.
Choose a small number of useful changes: correct wrong facts, improve a relevant listing, strengthen a service explanation or make the enquiry step clearer. Record the change date. Repeat the baseline and review enquiries alongside it; an improved answer alone does not prove additional revenue.
How often should we retest?
Set a rhythm that fits the business and the pace of change. Retain a fixed core and save new questions separately. When a service, area or question set changes materially, start a clearly dated new baseline. Comparing a new denominator with an old one can make progress look better or worse for the wrong reason.
How do I know whether my site is cited?
Open the source links in the saved answer and verify the destination. A source panel mentioning your domain and a direct citation supporting a claim should be recorded precisely rather than treated as interchangeable.
Does no citation mean the answer came only from model memory?
No. The absence of a visible source is not enough to determine how the answer was produced. Record what the interface exposes and avoid inventing a provenance you cannot inspect.
What is an AI visibility score?
It is a defined summary of observations. Ask for the questions, scope, evidence, denominator and unavailable cases. Different scoring methods should not be compared as though they use one standard scale.
Does a website access check measure recommendations?
No. It checks request responses, robots instructions and readable HTML under the stated conditions. Recommendation testing requires buyer questions and saved answers.
Can WhatBuyersAsk keep a baseline and export the evidence?
The portal selects 40 critical questions for the agreed research scope and retains the tested wording with the run. Completed research can provide raw saved answers and available scores with provenance. Review the actual models and methods rather than describing the export as clean consumer-chat sessions.
Give your next client review a clear foundation: the questions, the evidence and the next change worth making. Read how research works, create your workspace, or sign in to continue.
