We present a computational explainability approach for human comparison tasks, using Alignment Importance Score (AIS) heatmaps derived from deep-vision models. The AIS reflects a feature-map's unique contribution to the alignment between Deep Neural Network's (DNN) representational geometry and that of humans. We first validate the AIS by showing that prediction of out-of-sample human similarity judgments is improved when constructing representations using only higher-AIS feature maps identified from a training set. We then compute image-specific heatmaps that visually indicate the areas corresponding to feature-maps with higher AIS scores. These maps provide an intuitive explanation of which image areas are more important when it is compared to other images in a cohort. We observe a strong correspondence between these heatmaps and saliency maps produced by a gaze-prediction model. However, in some cases, meaningful differences emerge, as the dimensions relevant for comparison are not necessarily the most visually salient. In sum, Alignment Importance improves prediction of human similarity judgments from DNN embeddings, and provides interpretable insights into the relevant information in image space.

The posterior medial network is at the apex of a temporal integration hierarchy in the brain, integrating information over many seconds of viewing intact, but not scrambled, movies. This has been interpreted as an effect of temporal structure. Such structure in movies depends on preexisting event schemas, but temporal structure can also arise de novo from learning. Here, we examined the relative role of schema-consistent temporal structure and arbitrary but consistent temporal structure on the human posterior medial network. We tested whether, with repeated viewing, the network becomes engaged by scrambled movies with temporal structure. Replicating prior studies, activity in posterior medial regions was immediately locked to stimulus structure upon exposure to intact, but not scrambled, movies. However, for temporally structured scrambled movies, functional coupling within the network increased across stimulus repetitions, rising to the level of intact movies. Thus, temporal structure is a key determinant of network dynamics and function in the posterior medial network.