How do I measure AI citations without misleading myself?

Scope

Use this protocol to monitor whether public technical content appears, is linked, and supports generated answers across selected AI search or assistant products. The objective is reproducible observation and program learning. It is not reverse engineering and does not establish a universal market-share metric.

Why it happens

GEO results are easy to cherry-pick. A team tries several prompts, keeps the screenshot that cites the site, and compares it with a remembered baseline. Product mode, locale, login state, web access, conversation history, and time are unrecorded. The citation may not support the nearby claim, and no denominator is reported.

Dashboards create a different temptation. Citation counts are useful platform observations, but definitions, coverage, and query inventory are controlled by the provider. A rising count may reflect product rollout, demand, indexing, more pages, or content changes.

Diagnosis

Audit the current report. For every percentage or trend, find the numerator, denominator, question set, products, modes, observation dates, and missing data. Re-open sampled citations and classify support: exact, partial, topic-only, contradictory, stale, or unavailable.

Check whether questions changed after content was published. If a team replaces failed prompts with favorable variants, longitudinal comparison is invalid. New discovery questions can be added as a separate cohort without rewriting the baseline.

Solution

Create a stratified question set from real reader problems: informational, diagnostic, comparison, recommendation, and brand-aware queries. Freeze exact wording for the longitudinal lane and version any changes. Run repeated observations under documented conditions at a sustainable cadence.

Store answer snapshots or structured notes within product terms. For each source appearance, record page, claim, support classification, prominence, and whether the answer recommends, neutrally cites, or contradicts the source. Join available webmaster and referral data, keeping provider definitions separate.

Use content changes as hypotheses. Stagger updates where practical and compare matched pages or queries, but describe causal conclusions conservatively.

Artifact

The observation protocol records:

Group	Fields
Question	Stable ID, exact wording, intent, language, audience, and cohort
Run	Product, mode, account/login, locale, date/time, model if shown, and web state
Trial	Fresh session/history rule, sequence, errors, and answer artifact
Appearance	Mention, citation/link, cited URL, position/prominence, and surrounding claim
Support audit	Exact/partial/topic-only/contradictory/stale/unavailable plus reviewer
Accuracy	Correct book/author/title/claim and material omissions
Behavior	Referral, landing page, engagement event under privacy policy, and conversion boundary
Reporting	Numerator, denominator, missingness, uncertainty, product-specific definition, and date
Change log	Page update, hypothesis, comparison plan, and decision

Common mistakes

• Reporting the best screenshot from an undisclosed prompt search.
• Counting a source link without checking the claim it supports.
• Mixing product modes, countries, languages, and logged-in states in one rate.
• Treating a provider preview dashboard as a complete cross-platform measure.
• Claiming a page edit caused a change without stable questions or comparison evidence.