Proceedings of the Annual Meeting of the Cognitive Science Society

Humans coordinate by forming and committing to shared intentions—a form of shared agency that is largely absent in standard reinforcement learning agents and remains unclear in large language models, which function more like tools than autonomous agents. We compared human–human and human–AI collaborations using a non-linguistic sequential coordination game designed to disentangle successful collaboration outcomes from the underlying coordination mechanisms, such as commitment to an established joint goal despite the emergence of conflicting alternatives. Although adults and 7-year-old children were often able to adapt to different AI partners and achieve successful outcomes, human–human collaboration was marked by uniquely high levels of commitment and coordination efficiency, both of which were reduced in human–AI interactions. Five-year-old children showed greater difficulty collaborating with AI partners that functioned merely as asocial tools. Our findings suggest that human-compatible AI require mechanisms beyond outcome optimization to support efficient and stable collaboration.