Skip to main content
eScholarship
Open Access Publications from the University of California

Investigating Iconicity in Vision-and-Language Models: A Case Study of the Bouba/Kiki Effect in Japanese Models

Abstract

Extensive evidence from diverse areas of the cognitive sciences suggests that iconicity—the resemblance between form and its meaning—is pervasive and plays a pivotal role in the processing, memory, and evolution of human language. However, despite its acknowledged importance, iconicity in language models remains notably underexplored. This paper examines whether Japanese language models learn iconic associations between shape and sound, known as the bouba/kiki (or maluma/takete) effect, which has been widely observed in human language as well as English and multilingual vision-and-language models, including Finnish, Indonesian, Hungarian, and Lithuanian models in previous studies. A comparison between the current results and the previous studies revealed that Japanese models learn language-specific aspects of iconicity, such as the associations between /p/ and roundness, and /ɡ/ and hardness, reflecting the sound symbolic system in Japanese.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View