Stimulus recency has a strong effect on both behavior and neural responses. Its effects on neural responses have been most closely studied in the visual system in inferotemporal cortex (IT) in which recency gives rise to suppressed responses by a phenomenon known as repetition suppression. This observation has led to many possible explanations of how repetition suppression arises in the visual system. Here, we explore three of them: (1) top-down, (2) bottom-up and (3) independently in each brain region. Each of these accounts makes different predictions about the pattern of effects at different stages in visual processing for cases in which the stimulus either is or is not a match for the location or the identity of the preceding stimulus. We tested these predictions by recording from neurons in IT and V2, two separate stages of processing, while monkeys viewed displays of repeated and non-repeated image sequences.