Mild Traumatic brain injury (mTBI) is a signature wound in military personnel, and repetitive mTBI has been linked to age-related neurogenerative disorders that affect white matter (WM) in the brain. However, findings of injury to specific WM tracts have been variable and inconsistent. This may be due to the heterogeneity of mechanisms, etiology, and comorbid disorders related to mTBI. Non-negative matrix factorization (NMF) is a data-driven approach that detects covarying patterns (components) within high-dimensional data. We applied NMF to diffusion imaging data from military Veterans with and without a self-reported TBI history. NMF identified 12 independent components derived from fractional anisotropy (FA) in a large dataset (n = 1,475) gathered through the ENIGMA (Enhancing Neuroimaging Genetics through Meta-Analysis) Military Brain Injury working group. Regressions were used to examine TBI- and mTBI-related associations in NMF-derived components while adjusting for age, sex, post-traumatic stress disorder, depression, and data acquisition site/scanner. We found significantly stronger age-dependent effects of lower FA in Veterans with TBI than Veterans without in four components (q < 0.05), which are spatially unconstrained by traditionally defined WM tracts. One component, occupying the most peripheral location, exhibited significantly stronger age-dependent differences in Veterans with mTBI. We found NMF to be powerful and effective in detecting covarying patterns of FA associated with mTBI by applying standard parametric regression modeling. Our results highlight patterns of WM alteration that are differentially affected by TBI and mTBI in younger compared to older military Veterans.