- Main
How Far Does Vision Go? Interpretable Visual Statistics from Word-Linked Web Image Distributions Predict Multisensory Norms for Chinese Words
Abstract
Recent work reports that large language models, including those with visual training, fail to recover the sensorimotor structure of human concepts. We argue this reflects how visual information has been operationalized, as opaque embeddings tied to single training instances, rather than a limit of vision itself. We test this by representing 593 Chinese disyllabic words through interpretable low-level visual statistics, pixel-level and mid-level features aggregated over word-linked, CLIP-filtered web images, and evaluating their predictive power for human sensorimotor norms across six modalities beyond lexical controls. We find a graded profile of cross-modal decodability. Visual norms are predicted best, followed by gustatory, olfactory, tactile, and auditory, while interoception sits at the predicted lower bound. The graded pattern, with substantial gains for non-visual modalities, indicates that prior negative results reflect a choice of representation rather than a failure of low-level visual statistics per se.