A Nurse is Blue and Elephant is Rugby: Cross Domain Alignment in Large Language Models Reveal Human-like Patterns
Abstract
Cross-domain alignment refers to the task of mapping a concept from one domain to another, for example, ``If a \textit{doctor} were a \textit{color}, what color would it be?''. This seemingly peculiar task was designed to investigate how people represent concrete and abstract concepts through their mappings between categories and their reasoning processes over those mappings. In this paper, we adapt this task from cognitive science to evaluate the conceptualization and reasoning abilities of large language models (LLMs) through a behavioral study. We examine several LLMs by prompting them with a cross-domain mapping task and analyzing their responses at the population level and the individual level. Additionally, we assess the models' ability to reason about their predictions by analyzing and categorizing their explanations for these mappings. The results reveal several similarities between humans' and models' mappings and explanations, suggesting that models represent concepts similarly to humans. This similarity is evident not only at the model representation level but also in their behavior. Furthermore, the models mostly provide valid explanations and deploy reasoning paths that are similar to humans.