Proceedings of the Annual Meeting of the Cognitive Science Society

Convolutional neural networks (CNNs) have proven effective as models of visual semantic responses in the inferior tem-poral cortex (IT). The belief has been that training a network for visual recognition leads it to represent visual features in away similar to those the brain has learned. However, a CNNs response is affected by its architecture and not just its train-ing. We therefore explicitly measured the effect of training different CNN architectures on their representational similaritywith IT. We evaluated two versions of AlexNet and two training regimes, supervised and unsupervised. Surprisingly, wefound that the representations in an untrained (random-weight) variant of AlexNet, reflected brain representations in ITbetter than the benchmark supervised AlexNet and also better than the corresponding network trained in either a super-vised or unsupervised manner. These results require a re-evaluation of the explanation of why CNNs act as an effectivemodel of visual representations.