Institute for Mathematical Behavioral Sciences

The concept of surprise is central to sensory processing, adaptation and learning, attention, and decision making. Yet, no widely accepted mathematical theory currently exists to quantitatively characterize surprise elicited by a stimulus or event, for observers that range from single neurons to complex natural or engineered systems. We describe a formal Bayesian definition of surprise that is the only consistent formulation under minimal axiomatic assumptions. Surprise quantifies how data affects a natural or artificial observer, by measuring the difference between posterior and prior beliefs of the observer. Using this framework we measure the extent to which humans look towards surprising things while watching television and video games. We find that surprise is the strongest known attractor of human attention, with 72% of all human gaze shifts directed towards locations more surprising than on average, a figure which rises to 84% when considering only gaze targets simultaneously selected by four humans. The resulting theory of surprise is applicable across different modalities, data types, tasks, and abstraction levels.