The spatial components of a visual scene are processed neurally in a sequence of coarse features followed by fine features. This coarse-to-fine temporal stream was initially considered to be a cortical function, but has recently been demonstrated in the dorsal lateral geniculate nucleus. The goal of this study was to test the hypothesis that coarse-to-fine processing is present at earlier stages of visual processing in the retinal ganglion cells that supply lateral geniculate nucleus (LGN) neurons. To compare coarse-to-fine processing in the cat's visual system, we measured the visual responses of connected neuronal pairs from the retina and LGN, and separate populations of cells from each region. We found that coarse-to-fine processing was clearly present at the ganglion cell layer of the retina. Interestingly, peak and high-spatial-frequency cutoff responses were higher in the LGN than in the retina, indicating that there was a progressive cascade of coarse-to-fine information from the retina to the LGN to the visual cortex. The analysis of early visual pathway receptive field characteristics showed that the physiological response interplay between the center and surround regions was consistent with coarse-to-fine features and may provide a primary role in the underlying mechanism. Taken together, the results from this study provided a framework for understanding the emergence and refinement of coarse-to-fine processing in the visual system.