Being Cited Is Not Being Represented Correctly: Nearly Half of ChatGPT’s Auditable Citations Still Fail to Fully Support the Claim

Being Cited Is Not Being Represented Correctly: Nearly Half of ChatGPT’s Auditable Citations Still Fail to Fully Support the Claim
FREE PERSONALIZED ALERTS

Stay ahead of SEO & AI Search.

Follow the topics that matter to you. Get relevant new articles by email every 8 hours.

Create Free Alerts Free to join · Choose your topics · Unsubscribe anytime

Earning a ChatGPT citation is not the same thing as having the source represented faithfully. A live measurement project from Machine Relations is testing that distinction at the level where it actually matters: the individual claim and the citation placed beside it. In the current longitudinal series, only 53.6% of citations that the instrument can confidently map to a specific claim receive its strongest verdict, “Supported.”

The result comes from Machine Relations’ Citation Support Gap, a proprietary research instrument that collects and grades AI citation events and updates its collection-time series nightly. At the time of this article’s verification, the current partition had grown to 8,638 citation events, with 75.7% resolvable coverage and a 53.6% supported rate among those resolved pairs. The figures have already moved beyond the 7,858-event, 75.2% and roughly 53% snapshot observed on September 27, illustrating why this dataset should be treated as a live series rather than a frozen benchmark. citeturn0search0

The finding is striking, but it needs careful framing. Machine Relations is measuring its own defined prompt population with its own calibrated automated instrument. This is not an independent audit of all ChatGPT citations, and it does not establish that 46.4% of every citation ChatGPT produces is false. It shows that within the current auditable portion of this specific measurement series, nearly half do not meet the instrument’s strictest standard for fully supporting the claim they anchor. citeturn0search0

The first problem is whether a citation can be mapped to a claim at all

Before Machine Relations asks whether a source supports an assertion, it asks a more basic question: which assertion is the citation actually supposed to support?

The study calls a citation resolvable when it can be pinned to a specific claim without guessing. Citations attached to discrete statements are easier to audit. Citations dropped after paragraphs containing several factual assertions can be ambiguous because there is no reliable way to determine which statement the source is meant to substantiate. citeturn0search0

This is why the 75.7% coverage figure belongs beside the 53.6% support figure. The latter is not calculated across every citation event. It is calculated only across the subset the instrument can honestly resolve to a claim. citeturn0search0

For users, this creates two separate transparency problems. A citation can be difficult to attribute to a precise assertion, and even when the mapping is clear, the cited page may not fully support what the AI wrote.

“Supported” is deliberately a demanding category

Machine Relations does not grade citation fidelity as a simple true-or-false question. Its framework uses a five-label scale: Supported, Amplified, Contradicted, Misattributed and Fabricated, with Unreadable used operationally when the page cannot be fetched. citeturn0search0

“Supported” is the top category. A citation that points toward a broadly related source but stretches beyond what that source establishes can therefore fail to receive full credit without necessarily representing an outright fabrication.

That distinction is crucial. Saying that nearly half of auditable citations fail full support is not equivalent to saying nearly half are completely wrong. The remainder can contain several kinds of mismatch, including overstatement, attribution errors and stronger factual failure.

The live series closely echoes the earlier frozen ChatGPT baseline

The current result is notable partly because it is close to Machine Relations’ earlier frozen baseline. In that legacy corpus, ChatGPT had 1,630 citation events, of which 1,255—or 77.0%—could be resolved to a claim. Among those resolvable citations, 54.8% were graded Supported. citeturn0search0

The ongoing collection-time series currently sits at 53.6% supported among resolved events. The methodologies are not supposed to be pooled into one headline rate, and Machine Relations explicitly keeps the legacy baseline and longitudinal partitions separate. Still, the proximity of the figures means the larger live series has not so far produced a dramatic reversal of the earlier directional result. citeturn0search0

A September 19 analysis from Paralax similarly documented the frozen 54.8% ChatGPT figure and emphasized the importance of reporting support rates together with resolvable coverage rather than presenting an apparently universal “citation accuracy” percentage. citeturn0search11

The newer series fixes an important provenance weakness

Machine Relations is unusually explicit about a limitation in its original baseline. In the frozen corpus, cited webpages were fetched at grading time, sometimes days after the AI answer had been generated. A page could have changed during that interval. The baseline therefore measured whether the page supported the claim when it was later retrieved, not necessarily what the page contained at the exact moment ChatGPT cited it. citeturn0search0

The ongoing longitudinal series changes that design. From cycle 0001 onward, cited pages are snapshotted at collection time, minutes after the answer is produced, and grading uses that stored snapshot. citeturn0search0

This is a substantial methodological improvement because citation fidelity is inherently time-sensitive. A source that changes after an AI answer is generated can make a later audit unfair in either direction.

The denominator itself is versioned rather than silently blended

The live dataset also exposes another issue that many AI visibility dashboards hide: measurement rules can change. Machine Relations says the collection-time track is partitioned by a semantic key when denominator or evidence policy changes. Its current reported partition is labelled bounded-retry-defer, reflecting a change in how snapshot failures are retried before an event is finalized. citeturn0search0

Instead of combining events produced under different policies into one smoother trend line, the project reports each partition separately. That makes the headline less convenient but the comparison more defensible.

For GEO measurement, this is a useful lesson in its own right. A metric can move because the AI system changed, because the prompt basket changed, because the web changed or because the measurement pipeline changed. Good longitudinal reporting needs to separate those possibilities.

Gemini and Claude still do not have a comparable published support rate

The legacy cross-engine baseline demonstrates why coverage thresholds matter. Machine Relations reported resolvable coverage of 77.0% for ChatGPT and 47.6% for Perplexity, but only 9.0% for Gemini and 6.5% for Claude. It therefore withheld support rates for Gemini and Claude rather than calculating percentages from evidence it considered too sparse or ambiguous. citeturn0search0

This prevents a tempting but invalid leaderboard. A system whose citation style makes only a tiny fraction of citation-to-claim relationships auditable cannot be fairly compared with one whose inline citations are much easier to pin to discrete assertions.

Auditability is therefore a metric of its own, not merely a technical nuisance on the way to measuring support.

A citation can fail without the underlying source being low quality

The support gap should not be interpreted as a simple source-quality problem. A highly authoritative page can still be used incorrectly. The model can overgeneralize, combine evidence from multiple places, attach the wrong citation to a sentence or make an assertion stronger than the cited document warrants.

This separates two questions that AI-search analysis often merges: “Is this a good source?” and “Does this source support this particular sentence?”

The second question is much narrower. Citation fidelity is relational: it depends on the match between a generated claim and the evidence supplied for that claim.

Google research has found a different-sized support gap under a different method

The Machine Relations result should also not be generalized across engines. NetContentSEO recently covered a large academic measurement of Google AI Overviews in “Nearly 30% of Google AI Overview Sources Didn’t Rank on Page One—and 11% of Claims Lacked Citation Support.” That study decomposed AI Overview answers into 98,020 atomic factual claims and classified 11.0% as inconsistent with the cited content its pipeline could retrieve. citeturn0search1

The two percentages should not be compared as though they measure the same object. The Google study and Machine Relations use different systems, sampling, claim construction, labels and verification procedures. Their common lesson is narrower: displaying a citation is not proof that the adjacent generated statement is faithfully supported.

Citation visibility and citation fidelity are separate GEO outcomes

For publishers, the finding exposes a weakness in one of the most popular GEO metrics. Citation trackers typically ask whether a domain or URL appeared. That measures visibility, but not representation quality.

A publisher can “win” a citation and still lose control of the factual relationship between its source and the generated sentence. The AI can amplify a claim beyond what the page says, attribute an assertion incorrectly or place the citation beside a statement the source does not establish.

This extends the distinction NetContentSEO explored in “Your Page Can Be Cited by AI Without Actually Shaping the Answer—GEO Needs an ‘Absorption’ Metric.” Citation selection, answer influence and factual support are three different properties. A source can score differently on each. citeturn0search5

Publishers need representation monitoring, not only mention monitoring

The operational implication is that brands should inspect what the AI says around a citation, not merely record that the citation exists. For high-value prompts, monitoring can capture the generated claim, the cited URL, the relevant passage from the source and whether the relationship remains faithful over time.

This is particularly important for medical, financial, legal, product-specification and corporate information, where an apparently positive citation can still create risk if the AI exaggerates or misstates the source.

The goal is not to hold publishers responsible for every model error. It is to detect when visibility is producing a representation problem that ordinary citation-count dashboards would classify as success.

Clear, atomic evidence may make verification easier—but it cannot guarantee fidelity

Publishers can improve the conditions for accurate extraction by writing factual claims clearly, associating numbers with dates and units, identifying entities explicitly and keeping supporting evidence close to the assertion it substantiates. Those practices can reduce ambiguity for both humans and machines.

They cannot guarantee that an answer engine will use the material correctly. Citation assignment and synthesis happen downstream of the publisher’s page, inside systems the publisher does not control.

That is why citation fidelity should be monitored as an outcome rather than treated as something that can be solved entirely through page formatting.

The live number will keep moving

Machine Relations describes the collection-time series as a nightly-growing measurement. The figure observed on September 27—7,858 events, 75.2% resolvable coverage and roughly 53% supported among resolved citations—has already been superseded on the primary research page by 8,638 events, 75.7% coverage and 53.6% supported. citeturn0search0

That is not a contradiction; it is the intended behavior of a live monitor. It does mean articles, dashboards and social posts should attach a timestamp and denominator whenever they quote the rate.

A percentage without the cohort size, coverage and measurement date can quickly become stale or misleading.

Being cited is only the beginning of the audit

The emerging AI-search measurement stack now has several distinct questions. Was the page retrievable? Was it selected? Was it cited? Did it visibly shape the answer? And, finally, did the cited source actually support the claim attached to it?

Machine Relations’ current series focuses on that last question and produces an uncomfortable result: among the citation-to-claim pairs its instrument can resolve, only a little over half currently reach its strongest support category. citeturn0search0

The number should not be promoted as a universal ChatGPT accuracy rate, nor as an independent audit. But it is strong enough to expose a measurement mistake that publishers and AI users should avoid. A visible citation proves that a source was presented. It does not prove that the source was represented correctly.

0%