Category learning is a fundamental process in human cognition. Recent efforts have attempted to adapt theories developedin vision to the auditory domain. However, no study has directly compared auditory and visual category learning in thesame individuals. Using a fully within-subjects approach, we trained participants on non-speech auditory, visual, andnon-native speech categories in a single day. By comparing category learning behavior, the ability to generalize to novelcategory exemplars, and leveraging decision bound computational models, we found that while individuals demonstratedsimilar learning across the auditory and visual modalities, there were distinct perceptual biases that influenced learning ofnon-speech auditory categories. Further, there were substantial individual differences in performance across the three tasks.This study presents a novel comparison of category learning across modalities in the same individuals and demonstratesthat although commonalities exist, there is some domain-specificity to category learning.

Categorization is a fundamental cognitive skill that spans the senses. Even so, most research has focused on categorization and category learning in the visual modality. As a result, it is not yet clear how modality influences the perceptual and cognitive processes supporting category learning. In two experiments, we tested whether category learning results in amodal or modality-specific representations. We found strong evidence for modality-specific representations with independent learning across modalities. These results highlight the need to look beyond vision when constructing and testing models categorization and category learning. These findings also contribute to the longstanding debate on the amodal/modal nature of human knowledge, which is of broad interest to the cognitive science community.

Learning the sounds of a new language depends on the ability to learn novel auditory categories. Multidimensional categories, whether speech or nonspeech, can be learned through feedback and different category structures are proposed to recruit separate cognitive and neural mechanisms. There is substantial individual variability in learning; however, it is rare to compare learning of different categories in the same individuals. Understanding the sources of variability has theoretical implications for category learning. In this study, we trained the same participants on three types of multidimensional auditory categories. Participants learned nonspeech rule-based, nonspeech information-integration, and Mandarin speech categories. Learning of all category types was related across individuals and differences in working memory similarly supported learning across tasks. There was substantial variability in learning outcomes and strategies used to learn the categories. There are multiple paths to successful learning and appreciation of individual variability is essential to understanding the mechanisms of learning.

Learning to identify a person’s voice is a key component of speech perception. In this study, we use a categorization framework to provide insights about the mechanisms supporting talker identification. Native Mandarin Chinese listeners learned to categorize sentences in three tasks with different language contexts – native Mandarin talkers speaking Mandarin, native English talkers speaking English, and native Mandarin talkers speaking English. We compared learning when listeners received fully informative or minimal feedback. Using decision bound models, we examined the strategies participants used in each of the three tasks. Regardless of language context, full feedback was initially better for learning than minimal feedback but was no different after the second block. Across tasks, participants often used strategies based on mean fundamental frequency to separate the talkers. These results demonstrate that talker identification is a categorization problem, which enables leveraging existing category learning frameworks to understand the mechanisms of this important ability.

Category learning is a fundamental skill across modalities. Previous studies have investigated how children learn categories, primarily focusing on a single modality within a study. As a result, it is not well understood how the same children approach category learning tasks across modalities. In this study, we investigate 7–12-year-old children’s ability to learn rule-based or information-integration categories in the auditory and visual modalities. Our results indicate that children learn and generalize their knowledge better for visual than auditory categories, regardless of category type, and for rule-based than information-integration categories, regardless of modality. Even so, learning was strongly correlated across all tasks. Children overwhelmingly used unidimensional rule-based strategies to learn, regardless of whether it was optimal for the task. These results demonstrate that there are individual differences in children’s ability to learn perceptual categories across modalities and suggest that category learning in children is both category- and modality-general.

A popular dual systems theory of category learning argues thatthe structure of categories in perceptual space determines themechanisms that drive learning. However, less attention hasbeen paid to the nature of the perceptual dimensions definingthe categories. Researchers typically assume that there is adirect, linear relationship between experimenter-definedphysical input dimensions and learners’ psychologicaldimensions, but this assumption is not always warranted.Through a set of simulations, we demonstrate that, based on thenature of prior experience, the psychological representations ofexperimenter-defined dimensions can place drastic constraintson category learning. We compare the model’s behavior toseveral human studies and make conclusions regarding thenature of the psychological representations of the dimensionsin those studies. These simulations support the conclusion thatthe nature of psychological representations is a critical aspectto understanding the mechanisms that drive category learning.

Little is known about how categories are learned incidentallywithout instructions to group objects, overt decisions aboutcategory identity, or feedback about these decisions. Here weinvestigate how category learning may occur based on theassociation of categories with behaviorally-relevant events andactions. Previous research developed the SystematicMultimodal Associations Reaction Time (SMART) task inwhich participants report the location of a visual target with akeypress. The location of an upcoming visual target ispredicted by the identity of a novel sound category, exemplarsof which precede appearance of the visual target. Thiscategory-to-location mapping supports incidental learning ofauditory categories, with generalization to novel exemplars.Here, we examined whether this learning is driven by thecategory-to-location relationship, or instead by the associationwith distinct response alternatives. Across two experiments, weobserve that both a covert, reaction time measure of categorylearning and an overt labeling task testing generalization oflearning converge to indicate that the category-to-responserelationship drives incidental learning in the SMART task.