How do networks of neurons compute stable representations of continually transforming sense data? This thesis aims to provide a mathematical foundation for this question based in group theory -- the mathematics that describes much of the transformation structure in natural visual signals. The primary contribution of this thesis is a framework that ties group theory to neurophysiologically plausible computational mechanisms by way of generalized Fourier analysis and its roots in group representation theory. I demonstrate mathematically and computationally how group representations may be instantiated in biological neurons to achieve robust invariance and equivariance to signal transforms. I further demonstrate how complete, selective group invariance can be achieved in a third-order generalization of the classical energy model of complex cells, based on the bispectrum. This perspective offers a reframing and generalization of canonical models of receptive fields in visual cortex, explains extra-classical results, and makes novel empirical predictions. I additionally demonstrate the utility of this mathematical framework for building artificial neural networks with improved robustness and invariance and apply these architectures to the problems of learning latent transformation structure from data and group-invariant denoising of signals in associative memory networks.