UC San Diego

Human social intelligence is one of the defining features of our species, however, its origins and mechanisms are not well understood. The advent of Large Language Models (LLMs)—which learn to produce text on the basis of statistical patterns in the distribution of words—both threaten the uniqueness of human social intelligence and promise opportunities to better understand it. In this dissertation, I evaluate the extent to which distributional information learned by LLMs allows them to approximate human behavior on tasks that appear to require social intelligence. First, I compare human and LLM responses in experiments designed to measure theory of mind—the ability to represent and reason about the mental states of other agents. LLMs achieve parity with humans on some tasks (demonstrating that language statistics can in principle underpin mentalistic reasoning) but lag behind in others, suggesting that humans may rely on additional mechanisms. Second, I evaluate LLMs using the Turing test, which measures a machine's ability to imitate humans in a multi-turn social interaction. One model achieves a 50% pass rate, meaning participants are at chance in distinguishing it from a human. Collectively, the results suggest that LLMs simulate many aspects of our social intelligence, but by mechanisms that are potentially quite different from the ones that underpin human social cognition.