PAPER

When AI enters search, referrals fall — and so does user satisfaction

When AI answers enter search, do clicks to sites actually fall? And are users more satisfied in exchange?

AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence

Authors: Stephanie T. Wang (corresponding author) · Jeffrey Gleason · Yakov Bart · Christo Wilson · Danaé Metaxa
Affiliation: University of Pennsylvania · Northeastern University (supported by NSF IIS-2442711)
Venue: arXiv preprint (preregistered field experiment)
Submitted: 2026-08-18
arXiv: arXiv:2608.18352
We verified: 2026-08-24

WHAT THE PAPER SAYS

They fell, and satisfaction fell with them. In a preregistered field experiment (N=1,100), users assigned to AI Mode had an 18.8 percentage-point lower publisher click-through rate, while removing AI Overviews from the screen raised it by 8.8 points. Under the same conditions, trust, usefulness, satisfaction and sense of control all dropped significantly — the common claim that clicks fall but the user experience improves did not hold in this experiment.

Read the original on arXiv ↗

METHOD

How it was measured

Read the conditions before the numbers. The same figure means something different under a different sample or environment.

Design: Preregistered field experiment — the only paper in this library that supports a causal estimate
Sample: 1,444 recruited (Prolific 1,387 · Northeastern 57) → N=1,100 analysed (at least one search across seven days)
Period: Recruited 2026-03-17–19, seven days of treatment
Conditions: Assigned to AI Mode · AI Overviews removed · integrated (control)
Sample characteristics: 75% aged 45 or under · 87% college-educated or above · 58% Democrat / 20% Republican · 68% white — younger, more educated and more left-leaning than the US general population

FINDINGS

What came out

Publisher click-through rate: -18.8pp when assigned to AI Mode (95% CI -22.2, -15.3 · p<0.001). +8.8pp when AI Overviews were removed (95% CI 2.3, 15.3 · p=0.008)
Click decline by domain type (AI Mode): Advertising -42.7pp · Reddit -21.2pp · news sites -12.5pp · Wikipedia -9.9pp
User perception (AI Mode): Trust -0.34 points (7-point scale) · usefulness -0.59 SD · satisfaction -0.73 SD · sense of control -0.66 SD (all p<0.001)
Usage: -0.92 sessions per day when assigned to AI Mode (p<0.001). Removing AI Overviews produced no significant change

LIMITATIONS

Limitations the authors state themselves

Not our criticism — this is what the authors wrote in the paper.

In the AI-Overviews-removed condition, compliance fell to about 50% because of the interface change, so a LATE (local average treatment effect) estimate is reported — if the interface change altered the experience broadly, the exclusion restriction does not hold.
The sample is younger, more educated and more left-leaning than the US general population.
A seven-day treatment captures only short-term change and misses long-run adaptation.
Baseline AI Mode usage was very low at 0.6%, so this study reflects a **forced full-rollout scenario**. Effects under voluntary adoption may differ.

NAVIRANG'S READING — NOT THE PAPER'S CONCLUSION

This paper carries different weight from the rest of the library. Where the others observe what gets cited, this one randomly assigned conditions and measured causally what happens to traffic. Two parts matter in practice. First, advertising clicks fell hardest at -42.7pp — the structure of buying traffic with ad budget is the first thing to shake. Second, user satisfaction fell alongside, which means the familiar defence that 'AI is better for users so the click loss is acceptable' did not hold, at least in this experiment. That said, it is a US sample over seven days, and Korean-language or Naver environments were never in scope. We use this only as evidence that the traffic decline is real, and we do not carry its magnitudes over as expectations for clients here.

IN COMPARISON

Where it diverges from other papers

The claims-table entries this paper appears in.

When AI answers enter search, what happens to site traffic?

This paper's position — It falls — confirmed causally

If you want your own brand's numbers rather than a paper's

Every figure here came from someone else's sample. Send us a URL and we ask all 7 answer engines directly and measure yours.

Request a free AEO audit 070-4015-6848

We reply within one business day.