Proceedings of the Annual Meeting of the Cognitive Science Society

Do linguistic structures reflect the organization of human conceptual representations? A broad tradition in cognitive science argues that grammar is shaped by rich, compositional event representations that underlie cognition more broadly, motivating the use of linguistic patterns to probe the structure of thought. However, direct evidence linking linguistic and non-linguistic event cognition is sparse. Here, we test whether event categories are consistently represented across language and vision. Across two experiments, participants categorized event types using either linguistic descriptions or short video clips depicting the same classes of events. We find that humans reliably and consistently categorize events in both modalities, exhibiting parallel patterns of categorization across language and vision. Notably, this categorization is not inherent to every sophisticated cognitive system: State-of-the-art computational vision models failed at the same task. These results suggest that linguistic and visual event processing draw on a shared underlying system of event representation, providing evidence that language offers a window into human event cognition.