Proceedings of the Annual Meeting of the Cognitive Science Society

Hierarchical sequence processing is central to complex behaviours such as language, math, music, and tool use. Recent research has investigated hierarchical reasoning in both biological and artificial intelligences, but has failed to go beyond benchmarking. In this comparative cognition study, we tested hierarchical reasoning across 4 generations of state-of-the-art LLMs against 4 biological intelligences: adults, children, crows, and monkeys. We conducted Bayesian modelling analyses using both previously published in vivo data and newly collected in silico data. We found that the capacity for hierarchical reasoning is present across generations of LLMs, on a continuum that parallels biological intelligences. Newer (reasoning) models exhibited the strongest proclivity for hierarchical reasoning, while the more primitive ones showed graded performance mirroring that of children and non-humans. Together, these findings position LLM as a unique model organism for comparing hierarchical cognition across biological and artificial intelligences.