The visual system is tasked with processing massive amounts of sensory information, but its computational power is constrained by limited metabolic resources. This results in the need for selective filtering of inputs so that the most relevant information is highlighted for further processing. This selectivity can be implemented slowly over the course of visual system evolution and development, by adapting the tuning properties of sensory neurons to optimally represent the stimuli that are most likely to be encountered during natural behavior. It can also be implemented more rapidly during the behavior of an individual organism, as when the brain enhances representations of target objects that are known to be relevant in a given context. Finally, selectivity can be implemented in the memory system, by adaptively re-formatting remembered information according to the demands of a particular task. In this dissertation, I will present three complementary experiments that exemplify the role of selective, efficient information processing in shaping visual system function.