The cerebellar cortex has a key role in generating predictive sensorimotor associations. To do so, the granule cell layer is thought to establish unique sensorimotor representations for learning. However, how this is achieved and how granule cell population responses contribute to behavior have remained unclear. To address these questions, we have used in vivo calcium imaging and granule cell-specific pharmacological manipulation of synaptic inhibition in awake, behaving mice. These experiments indicate that inhibition sparsens and thresholds sensory responses, limiting overlap between sensory ensembles and preventing spiking in many granule cells that receive excitatory input. Moreover, inhibition can be recruited in a stimulus-specific manner to powerfully decorrelate multisensory ensembles. Consistent with these results, granule cell inhibition is required for accurate cerebellum-dependent sensorimotor behavior. These data thus reveal key mechanisms for granule cell layer pattern separation beyond those envisioned by classical models.