Proceedings of the Annual Meeting of the Cognitive Science Society

Deep neural networks have become increasingly successful atsolving classic perception problems such as object recognition,semantic segmentation, and scene understanding, often reach-ing or surpassing human-level accuracy. This success is duein part to the ability of DNNs to learn useful representationsof high-dimensional inputs, a problem that humans must alsosolve. We examine the relationship between the representa-tions learned by these networks and human psychological rep-resentations recovered from similarity judgments. We find thatdeep features learned in service of object classification accountfor a significant amount of the variance in human similarityjudgments for a set of animal images. However, these fea-tures do not capture some qualitative distinctions that are a keypart of human representations. To remedy this, we develop amethod for adapting deep features to align with human sim-ilarity judgments, resulting in image representations that canpotentially be used to extend the scope of psychological exper-iments.