Skip to main content
eScholarship
Open Access Publications from the University of California

A Rational Analysis of the Effects of Sycophantic AI

Creative Commons 'BY' version 4.0 license
Abstract

People increasingly use large language models (LLMs) to explore ideas, gather information, and make sense of the world. In these interactions, users encounter chatbots that are overly agreeable. We argue this sycophancy poses an epistemic risk distinct from hallucination: it distorts belief not by introducing falsehoods but by biasing the evidence users see. A rational analysis shows that a Bayesian agent fed examples sampled from its own hypothesis grows more confident in that hypothesis without moving closer to the truth. We tested this prediction in a modified Wason 2-4-6 rule discovery task where participants (N=557) interacted with AI agents providing different types of feedback. Unmodified LLM behavior suppressed participants' discovery and inflated their confidence comparably to explicitly sycophantic prompting. By contrast, independent sampling from the true distribution yielded discovery rates five times higher. This paper documents how sycophantic AI distorts belief, manufacturing certainty where there should be doubt.