From seeing to understanding: Novice sign language learners shift focus from perceptual to semantic information for newly learned signs
Abstract
Comprehending content in a newly learned language requires interactions between perceptual, semantic, and executive processing systems. Learners whose target language differs from their own in modality (e.g. spoken language users learning to sign) provide a unique opportunity to examine the relative contributions of semantic and perceptual processing. We present data from three studies where hearing, non-signing participants with between a few hours and a few weeks of experience with American Sign Language (ASL) viewed videos of signs during fMRI scanning. Using Representational Similarity Analysis (RSA), we measure contributions of semantic-conceptual, visual perception, and cognitive control regions to processing of semantic and visual information in ASL. Across different learning paradigms, including both cross-sectional and longitudinal approaches, we find that semantic features become more decodable in semantic, visual, and cognitive control regions after learning, while visual features become less salient, especially in visual cortex.