What are the roles of central and peripheral vision in human scene recognition? Larson and Loschky (2009) showed that peripheral vision contributes more than central vision in obtaining maximum scene recognition accuracy. However, central vision is more efficient for scene recognition than peripheral, based on the amount of visual area needed for accurate recognition. In this study, we model and explain the results of Larson and Loschky (2009) using a neurocomputational modeling approach. We show that the advantage of peripheral vision in scene recognition, as well as the efficiency advantage for central vision, can be replicated using state-of-the-art deep neural network models. In addition, we propose and provide support for the hypothesis that the peripheral advantage comes from the inherent usefulness of peripheral features. This result is consistent with data presented by Thibaut, Tran, Szaffarczyk, and Boucart (2014), who showed that patients with central vision loss can still categorize natural scenes efficiently. Furthermore, by using a deep mixture-of-experts model ("The Deep Model," or TDM) that receives central and peripheral visual information on separate channels simultaneously, we show that the peripheral advantage emerges naturally in the learning process: When trained to categorize scenes, the model weights the peripheral pathway more than the central pathway. As we have seen in our previous modeling work, learning creates a transform that spreads different scene categories into different regions in representational space. Finally, we visualize the features for the two pathways, and find that different preferences for scene categories emerge for the two pathways during the training process.

It is commonly believed that the central visual field (fovea andparafovea) is important for recognizing objects and faces, andthe peripheral region is useful for scene recognition. However,the relative importance of central versus peripheral informa-tion for object, scene, and face recognition is unclear. Larsonand Loschky (2009) investigated this question in the context ofscene processing using experimental conditions where a cir-cular region only reveals the central visual field and blocksperipheral information (”Window”), and in a ”Scotoma” con-dition, where only the peripheral region is available. Theymeasured the scene recognition accuracy as a function of vi-sual angle, and demonstrated that peripheral vision was indeedmore useful in recognizing scenes than central vision in termsof achieving maximum recognition accuracy. In this work,we modeled and replicated the result of Larson and Loschky(2009), using deep convolutional neural networks (CNNs).Having fit the data for scenes, we used the model to predictfuture data for large-scale scene recognition as well as for ob-jects and faces. Our results suggest that the relative order ofimportance of using central visual field information is facerecognition>object recognition>scene recognition, and vice-versa for peripheral information. Furthermore, our results pre-dict that central information is more efficient than peripheralinformation on a per-pixel basis across all categories, which isconsistent with Larson and Loschky’s data.

Are face and object recognition abilities independent? Although it is commonly believed that they are, Gauthier et al. [Gauthier, I., McGugin, R. W., Richler, J. J., Herzmann, G., Speegle, M., & VanGulick, A. E. Experience moderates overlap between object and face recognition, suggesting a common ability. Journal of Vision, 14, 7, 2014] recently showed that these abilities become more correlated as experience with nonface categories increases. They argued that there is a single underlying visual ability, v, that is expressed in performance with both face and nonface categories as experience grows. Using the Cambridge Face Memory Test and the Vanderbilt Expertise Test, they showed that the shared variance between Cambridge Face Memory Test and Vanderbilt Expertise Test performance increases monotonically as experience increases. Here, we address why a shared resource across different visual domains does not lead to competition and to an inverse correlation in abilities? We explain this conundrum using our neurocomputational model of face and object processing ["The Model", TM, Cottrell, G. W., & Hsiao, J. H. Neurocomputational models of face processing. In A. J. Calder, G. Rhodes, M. Johnson, & J. Haxby (Eds.), The Oxford handbook of face perception. Oxford, UK: Oxford University Press, 2011]. We model the domain general ability v as the available computational resources (number of hidden units) in the mapping from input to label and experience as the frequency of individual exemplars in an object category appearing during network training. Our results show that, as in the behavioral data, the correlation between subordinate level face and object recognition accuracy increases as experience grows. We suggest that different domains do not compete for resources because the relevant features are shared between faces and objects. The essential power of experience is to generate a "spreading transform" for faces (separating them in representational space) that generalizes to objects that must be individuated. Interestingly, when the task of the network is basic level categorization, no increase in the correlation between domains is observed. Hence, our model predicts that it is the type of experience that matters and that the source of the correlation is in the fusiform face area, rather than in cortical areas that subserve basic level categorization. This result is consistent with our previous modeling elucidating why the FFA is recruited for novel domains of expertise [Tong, M. H., Joyce, C. A., & Cottrell, G. W. Why is the fusiform face area recruited for novel categories of expertise? A neurocomputational investigation. Brain Research, 1202, 14-24, 2008].

To recognize objects, the human visual system processes information through a network of hierarchically organized brain regions. Many neurocomputational models have modeled this hierarchical structure, but they have often used hand-crafted features to model early visual areas. According to the linear efficient coding hypothesis, the goal of the early visual pathway is to capture the statistical structure of sensory stimuli, removing redundancy, and factoring the input into independent features. In this work, we use a hierarchical Independent Components Analysis (ICA) algorithm to automatically learn the visual features that account for early visual cortex. We then continue modeling the object recognition pathway using Gnostic Fields, a theory for how the brain does object categorization, in which brain regions devoted to classifying mutually-exclusive categories exist near the top of sensory processing hierarchies. The whole biologically-inspired model not only allows us to develop representations similar to those in primary visual cortex, but also to perform well on standard computer vision object recognition benchmarks.