Skip to main content
eScholarship
Open Access Publications from the University of California

Language as a bootstrap for compositional visual reasoning

Creative Commons 'BY' version 4.0 license
Abstract

People think and learn abstractly and compositionally. These two key properties of human cognition are shared with natural language: we use a finite, composable vocabulary of nameable concepts to generate and understand a combinatorially large space of new sentence. In this paper, we present a domain of compositional reasoning tasks and an artificial language learning paradigm designed to probe the role language plays in bootstrapping learning. We discuss results from a language-guided program learning model suggesting that language can play an important role in bootstrapping learning by providing an important signal for search on individual problems, and a cue towards named, reusable abstractions across the domain as a whole. We evaluate adults on the same domain, comparing learning performance between those tasked with jointly learning language and solving reasoning tasks, and those who only approach the domain as a collection of inductive reasoning problems. We find that adults provided with abstract language prompts are better equipped to generalize and compose concepts learned across a domain than adults solving the same problems using reasoning alone.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View