Proceedings of the Annual Meeting of the Cognitive Science Society

Modern deep neural networks require a tremendous amountof data to train, often needing hundreds or thousands of la-beled examples to learn an effective representation. For thesenetworks to work with less data, more structure must be builtinto their architectures or learned from previous experience.The learned weights of convolutional neural networks (CNNs)trained on large datasets for object recognition contain a sub-stantial amount of structure. These representations have par-allels to simple cells in the primary visual cortex, where re-ceptive fields are smooth and contain many regularities. In-corporating smoothness constraints over the kernel weightsof modern CNN architectures is a promising way to improvetheir sample complexity. We propose a smooth kernel regu-larizer that encourages spatial correlations in convolution ker-nel weights. The correlation parameters of this regularizer arelearned from previous experience, yielding a method with ahierarchical Bayesian interpretation. We show that our corre-lated regularizer can help constrain models for visual recogni-tion, improving over an L2 regularization baseline.