Categorizing spatial relations is central to the development of visual understanding and spatial cognition, with roots in the first few months of life. Quinn (2003) reviews two findings in infant relation categorization: categorizing one object as above/below another precedes categorizing an object as between other objects, and categorizing relations over specific objects predates abstract relations over varying objects. We model these phenomena with deep neural networks, including contemporary architectures specialized for relational learning and vision models pretrained on baby headcam footage (Sullivan et al., 2020). Across two computational experiments, we can account for most of the developmental findings, suggesting these neural network models are useful for studying the computational mechanisms of infant categorization.

People ask questions that are far richer, more informative, and more creative than current AI systems. We propose a neuro-symbolic framework for modeling human question asking, which represents questions as formal programs and generates programs with an encoder-decoder based deep neural network. From extensive experiments using an information-search game, we show that our method can predict which questions humans are likely to ask in unconstrained settings. We also propose a novel grammar-based question generation framework trained with reinforcement learning, which is able to generate creative questions without supervised human data.

Humans are highly efficient learners, with the ability to grasp the meaning of a new concept from just a few examples. Unlike popular computer vision systems, humans can flexibly leverage the compositional structure of the visual world, understanding new concepts as combinations of existing concepts. In the current paper, we study how people learn different types of visual compositions, using abstract visual forms with rich relational structure. We find that people can make meaningful compositional generalizations from just a few examples in a variety of scenarios, and we develop a Bayesian program induction model that provides a close fit to the behavioral data. Unlike past work examining special cases of compositionality, our work shows how a single computational approach can account for many distinct types of compositional generalization.

Systematic generalization is the ability to combine known parts into novel meaning; an important aspect of efficient human learning, but a weakness of neural network learning. In this work, we investigate how two well-known modeling principles --- modularity and data augmentation --- affect systematic generalization of neural networks in grounded language learning. We analyze how large the vocabulary needs to be to achieve systematic generalization and how similar the augmented data needs to be to the problem at hand. Our findings show that even in the controlled setting of a synthetic benchmark, achieving systematic generalization remains very difficult. After training on an augmented dataset with almost forty times more adverbs than the original problem, a non-modular baseline is not able to systematically generalize to a novel combination of a known verb and adverb. When separating the task into cognitive processes like perception and navigation, a modular neural network is able to utilize the augmented data and generalize more systematically, achieving 70% and 40% exact match increase over state-of-the-art on two gSCAN tests that have not previously been improved. We hope that this work gives insight into the drivers of systematic generalization, and what we still need to improve for neural networks to learn more like humans do.

The ability to generate rules and hypotheses plays a key role in multiple aspects of human cognition including concept learning and explanation. Previous research has framed this ability as a form of inference via probabilistic program induction. However, this modeling approach often requires careful construction of the right grammar and hypothesis space for a particular task, and cannot easily be transferred to other domains. In this work, we present an alternative computational account of rule generation, leveraging advances in multimodal learning and large language models. Taking naturalistic images as input, our computational model is capable of generating candidate rules that are specified in natural language, and verifying them to determine their fit to the data. We show that our model can generate, in a zero-shot manner, plausible rules for visual concepts across multiple domains.

People use rich prior knowledge about the world in order toefficiently learn new concepts. These priors–also known as“inductive biases”–pertain to the space of internal models con-sidered by a learner, and they help the learner make inferencesthat go beyond the observed data. A recent study found thatdeep neural networks optimized for object recognition developthe shape bias (Ritter et al., 2017), an inductive bias possessedby children that plays an important role in early word learning.However, these networks use unrealistically large quantities oftraining data, and the conditions required for these biases to de-velop are not well understood. Moreover, it is unclear how thelearning dynamics of these networks relate to developmentalprocesses in childhood. We investigate the development andinfluence of the shape bias in neural networks using controlleddatasets of abstract patterns and synthetic images, allowing usto systematically vary the quantity and form of the experienceprovided to the learning algorithms. We find that simple neuralnetworks develop a shape bias after seeing as few as 3 exam-ples of 4 object categories. The development of these biasespredicts the onset of vocabulary acceleration in our networks,consistent with the developmental process in children.

How do children learn correspondences between the language and the world from noisy, ambiguous, naturalistic input?One hypothesis is via cross-situational learning: tracking words and their possible referents across multiple situationsallows learners to disambiguate correct word-referent mappings (Yu and Smith, 2007). While previous models of cross-situational word learning operate on highly simplified representations, recent advances in multimodal learning have shownpromise as richer models of cross-situational word learning to enable learning the meanings of words from raw inputs.Here, we present a neural network model of cross-situational word learning that leverages some of these ideas and examineits ability to account for a variety of empirical phenomena from the word learning literature.