Proceedings of the Annual Meeting of the Cognitive Science Society

Visual scene understanding often requires the processing of human-object interactions. Here we seek to explore if andhow well Deep Neural Network (DNN) models capture features similar to the brain’s representation of humans, objects,and their interactions. We investigate brain regions which process human-, object-, or interaction-specific information, andestablish correspondences between them and DNN features. Our results suggest that we can infer the selectivity of theseregions to particular visual stimuli using DNN representations. We also map features from the DNN to the regions, thuslinking the DNN representations to those found in specific parts of the visual cortex. In particular, our results suggest thata typical DNN representation contains encoding of compositional information for human-object interactions which goesbeyond a linear combination of the encodings for the two components, thus suggesting that DNNs may be able to modelthis important property of biological vision.