Skip to main content
eScholarship
Open Access Publications from the University of California

Language as a bootstrap for compositional visual reasoning

  • Author(s): Wong, Catherine;
  • Friedman, Yoni;
  • Andreas, Jacob;
  • Tenenbaum, Josh
  • et al.
Abstract

People think and learn abstractly and compositionally. These two key properties of human cognition are shared with natural language: we use a finite, composable vocabulary of nameable concepts to generate and understand a combinatorially large space of new sentence. In this paper, we present a domain of compositional reasoning tasks and an artificial language learning paradigm designed to probe the role language plays in bootstrapping learning. We discuss results from a language-guided program learning model suggesting that language can play an important role in bootstrapping learning by providing an important signal for search on individual problems, and a cue towards named, reusable abstractions across the domain as a whole. We evaluate adults on the same domain, comparing learning performance between those tasked with jointly learning language and solving reasoning tasks, and those who only approach the domain as a collection of inductive reasoning problems. We find that adults provided with abstract language prompts are better equipped to generalize and compose concepts learned across a domain than adults solving the same problems using reasoning alone.

Main Content
For improved accessibility of PDF content, download the file to your device.
Current View