The Abstraction and Reasoning Challenge (ARC) is a set of tasks where one must induce a program from a few given input-output examples, and apply it to a new input. Although humans can easily solve most tasks, ARC is challenging for state-of-the-art algorithms. We hypothesize that humans use intuitive program induction, and interpret ARC tasks by constructing "natural programs'' in language. We experimentally study "natural programs'' by formulating a two-player game: A participant solves and then communicates the program to another participant using natural language; the second participant must solve the task using the description alone. We find at least 361 out of 400 tasks can be solved from a natural language description, demonstrating that natural language is sufficient in transmitting these natural programs. We compared the natural language programs to computer programs constructed using two separate state-of-the-art program synthesis approaches, and conducted a study on leveraging natural language annotations to improve performance of program-synthesis tools.

Core knowledge concepts such as object behavior principles provide a rich inventory of primitives for thinking and learning in the natural world. However, it remains unexplored how these concepts are reused for problem-solving and communication in highly abstract domains. We analyze a large-scale natural language study drawing on the Abstraction and Reasoning Corpus (ARC), a set of highly abstract visual tasks where solvers construct outputs from input grids according to an inferred pattern. ARC explicitly incorporates core knowledge principles without any real-world objects. In the study, subjects solved and communicated the inferred patterns of ARC tasks via written explanations for other subjects attempting to solve tasks using only the explanations. We examine how subjects solve, communicate, and interpret these explanations, and we show that subjects use fundamentally abstract core knowledge properties–object cohesion and contact causality–to reason about, understand, and communicate the inference tasks with language.