Concussion, or mild traumatic brain injury (mTBI), is a major public health concern, linked with persistent post-concussive syndrome, and chronic traumatic encephalopathy. At present, standard clinical imaging fails to reliably detect traumatic axonal injury associated with concussion and post-concussive symptoms. Diffusion tensor imaging (DTI) is an MR imaging technique that is sensitive to changes in white matter microstructure. Prior studies using DTI did not jointly investigate white matter microstructure in athletes, a population at high risk for concussive and subconcussive head traumas, with those in typical emergency room (ER) patients. In this study, we determine DTI scalar metrics in both ER patients and scholastic athletes who suffered concussions and compared them to those in age-matched healthy controls. In the early subacute post-concussion period, athletes demonstrated an elevated rate of regional decreases in axial diffusivity (AD) compared to controls. These regional decreases of AD were especially pronounced in the cerebellar peduncles, and were more frequent in athletes compared to the ER patient sample. The group differences may indicate differences in the mechanisms of the concussive impacts as well as possible compound effects of cumulative subconcussive impacts in athletes. The prevalence of white matter abnormality in cerebellar tracts lends credence to the hypothesis that post-concussive symptoms are caused by shearing of axons within an attention network mediated by the cerebellum, and warrant further study of the correlation between cerebellar DTI findings and clinical, neurocognitive, oculomotor, and vestibular outcomes in mTBI patients.