For publishers trying to understand what Google's shift toward generative search means for referral traffic, a new study offers something unusually valuable: randomized evidence from people using Google in their everyday browsing rather than another before-and-after traffic chart. In a preregistered field experiment involving 1,100 active participants, researchers found that forcing searches into Google AI Mode reduced click-through to external sites by 18.8 percentage points compared with the normal Google Search experience. The same intervention also reduced daily search sessions and pushed more users toward competing search engines.
The preprint, submitted to arXiv on August 18, was written by researchers affiliated with the University of Pennsylvania and Northeastern University. It has not been peer-reviewed, so its estimates should be treated as experimental evidence rather than a settled measurement of AI Mode's impact across Google's entire user base. Still, its design makes it materially different from most studies of AI search traffic: the researchers randomly assigned real Google users to different search experiences and measured their behavior in the field.
A randomized experiment inside everyday Google Search
Participants first completed a three-day baseline period and were then randomly assigned for seven days to one of three conditions. The control group continued using current Google Search without modification. A No AI Search group had AI Overviews hidden when the researchers' browser extension successfully detected them and had attempts to enter AI Mode redirected back to general search. The third group was routed into AI Mode for its Google searches.
The experiment enrolled 1,444 people, with 1,100 making at least one search during the seven-day treatment period and forming the behavioral analysis sample. The study population was restricted to U.S.-based adults who primarily used Chrome and Google Search. It also skewed younger and more highly educated than the U.S. population, an important limitation when interpreting how broadly the results generalize.
The AI Mode intervention worked consistently: 94.7% of searches in that group were successfully routed into AI Mode. The No AI intervention was less clean. A Google HTML change during the study broke the extension's AI Overview detection, meaning only 51.1% of AI Overviews were successfully hidden overall. The researchers therefore report intent-to-treat estimates for AI Mode and local average treatment effects for exposure to the No AI condition.
AI Mode cut external click-through by 18.8 percentage points
The headline result is the effect on external traffic. Assignment to AI Mode reduced click-through rate by 18.8 percentage points relative to current Google Search, with a 95% confidence interval from -22.2 to -15.3 percentage points. The result was statistically significant at p<0.001. Exposure to the No AI Search condition moved in the opposite direction, increasing external CTR by 8.8 percentage points, with a 95% confidence interval from 2.3 to 15.3 points.
The traffic effect was not confined to one type of publisher. AI Mode reduced the fraction of users clicking through to news sites by 12.5 percentage points, Reddit by 21.2 points and Wikipedia by 9.9 points. The study also recorded a 42.7-point decline in the fraction clicking ads, although that result needs a major qualification: AI Mode did not surface ads during the experiment, so it should not be treated as an estimate of how today's monetized implementations would necessarily behave.
These results complement a growing body of evidence that Google's generative surfaces can change where attention flows. NetContentSEO recently covered a separate study finding that the estimated AI Overview effect on Wikipedia traffic was revised from roughly 15% to about 5% after researchers changed their measurement design. That episode is a reminder that causal estimates depend heavily on methodology. The new AI Mode experiment is noteworthy precisely because random assignment provides a stronger basis for attributing observed behavioral differences to the search experience itself.
Users searched less often—and some tried rival engines
The experiment did not find evidence that forcing users into AI Mode generated more Google search activity. Instead, assignment to AI Mode reduced search sessions by 0.92 sessions per day, with a 95% confidence interval from -1.30 to -0.55. At the same time, sessions that did occur became 0.43 minutes longer on average. The combination suggests that participants spent more time inside individual AI-mediated sessions while initiating fewer Google search sessions overall.
One of the most striking secondary findings concerns search-engine substitution. Assignment to AI Mode increased the fraction of users who searched with Bing, DuckDuckGo or Yahoo by 11.2 percentage points, with a 95% confidence interval from 6.4 to 16.0 points. It also increased stated intention to switch to Bing by 1.25 points on a seven-point scale. The No AI Search intervention, by contrast, produced no detectable increase in competitor-engine use.
This result matters because AI Mode is often discussed primarily as a publisher-traffic issue. The experiment suggests that interface changes can also affect Google's own relationship with users. It does not establish that AI Mode will cause an 11.2-point increase in competitor usage across the general population, but within this randomized sample, forcing the AI-first experience measurably increased substitution toward other search engines.
The user-experience results challenge a simple traffic-for-convenience tradeoff
A common defense of zero-click search is that users may not need to visit an external page if the search engine has already answered the question satisfactorily. That can create a genuine tradeoff: fewer publisher referrals in exchange for faster task completion. The new experiment is interesting because its survey results did not show a corresponding improvement in user experience under forced AI Mode.
Participants assigned to AI Mode reported lower trust in information found on Google, with trust declining by 0.34 points on a seven-point scale. The researchers also found statistically significant reductions in perceived usefulness, satisfaction, agency and personalization or relevance. Satisfaction fell by 0.73 standard deviations, perceived agency by 0.66 standard deviations and usefulness by 0.59 standard deviations. By contrast, exposure to the No AI Search condition produced no detectable deterioration in these measured experience outcomes even while external CTR increased.
That distinction is important. The experiment does not prove that users universally dislike generative search, nor does it show that every AI-assisted query is inferior to conventional results. Qualitative feedback was mixed: 33.6% of AI Mode respondents expressed negative sentiment, 29.3% positive sentiment and 13.7% mixed sentiment. Some participants valued efficiency and summarization, while others reported reduced control, difficulty reaching specific websites, limited source diversity and concerns about accuracy.
What this means for publishers
For SEO teams, the study strengthens the case for separating two questions that are often blended together: whether a brand can appear inside an AI-generated answer and whether that appearance produces a visit. Visibility inside generative search may have value, but the experiment indicates that an AI-first interface can substantially reduce the probability that a user leaves Google for an external source. That makes citation visibility and referral traffic related but increasingly distinct metrics.
NetContentSEO has already documented evidence that Google's AI surfaces draw from a source ecosystem that differs from conventional organic rankings. One large study found that 29.8% of domains cited by AI Overviews did not appear on the co-displayed first page of organic results. Another 11,500-query analysis found that Google Search and AI Overviews shared only 18% of their sources. The new field experiment adds a behavioral layer to that picture: even when AI search surfaces information derived from the web, users may be materially less likely to continue to the web itself.
That does not make traditional SEO irrelevant. It makes measurement more complicated. Publishers increasingly need to distinguish classic organic ranking, AI citation or mention visibility, external referral traffic and downstream business outcomes rather than treating them as interchangeable expressions of search performance. A site can gain exposure inside an AI response while losing sessions, and the economic value of that exposure will depend on whether users later recognize the brand, convert through another channel or never leave the search interface at all.
The study is unusually strong, but it still has limits
The randomized design supports causal interpretation within the experiment, but it does not eliminate questions about external validity. Participants were U.S.-based Chrome users who already relied on Google as their primary search engine, and the sample was younger and more educated than the overall U.S. population. The intervention also forced AI Mode much more aggressively than ordinary users would necessarily choose it: during the baseline period, participants used AI Mode for only 0.6% of searches, while the treatment routed nearly all eligible searches there.
That makes the AI Mode condition best understood as an experiment measuring what happens when the conversational interface replaces conventional Google Search for these users, not as a direct estimate of today's average traffic loss across all Google searches. Product design is also moving quickly, meaning later versions of AI Mode may differ in linking behavior, interface quality, advertising and source presentation from the version participants encountered in March 2026.
Even with those caveats, the paper provides a rare piece of evidence in a debate dominated by platform claims, publisher analytics and observational studies. In this experiment, forcing Google users into AI Mode caused substantially fewer external clicks, fewer daily Google search sessions, lower measured satisfaction and trust, and more use of rival search engines. Removing AI features moved external CTR in the opposite direction without a measurable penalty on the user-experience outcomes the researchers tested.
For publishers, the 18.8-point figure is not a universal forecast. It is something more useful: a causal estimate from a clearly defined randomized intervention. As Google continues moving Search toward conversational answers, experiments like this offer a way to measure not merely what AI search displays, but how people actually change their behavior when the interface changes around them.