The sensory periphery is responsible for detecting ethologically relevant features of the external world, using compact, predominantly feedforward circuits. Visual motion is a particularly prevalent sensory feature, the presence of which can be a signal to enact diverse behaviors ranging from gaze stabilization reflexes to predator avoidance or prey capture. To understand how the retina constructs the distinct neural representations required for these behaviors, we investigated two circuits responsible for encoding different aspects of image motion: ON and ON-OFF direction-selective ganglion cells (DSGCs). Using a combination of two-photon targeted whole-cell electrophysiology, pharmacology, and conditional knockout mice, we show that distinct inhibitory pathways independently control tuning for motion velocity and motion direction in these two cell types. We further employ dynamic clamp and numerical modeling techniques to show that asymmetric inhibition provides a velocity-invariant mechanism of directional tuning, despite the strong velocity dependence of classical models of direction selectivity. We therefore demonstrate that invariant representations of motion features by inhibitory interneurons act as computational building blocks to construct distinct, behaviorally relevant signals at the earliest stages of the visual system.