Proceedings of the Annual Meeting of the Cognitive Science Society

The ability to generate rules and hypotheses plays a key role in multiple aspects of human cognition including concept learning and explanation. Previous research has framed this ability as a form of inference via probabilistic program induction. However, this modeling approach often requires careful construction of the right grammar and hypothesis space for a particular task, and cannot easily be transferred to other domains. In this work, we present an alternative computational account of rule generation, leveraging advances in multimodal learning and large language models. Taking naturalistic images as input, our computational model is capable of generating candidate rules that are specified in natural language, and verifying them to determine their fit to the data. We show that our model can generate, in a zero-shot manner, plausible rules for visual concepts across multiple domains.