The contribution of inflammation to deleterious aging outcomes is increasingly recognized; however, little is known about the complex relationship between interleukin-6 (IL-6) and brain structure, or how this association might change with increasing age. We examined the association between IL-6, white matter integrity, and cognition in 151 community dwelling older adults, and tested whether age moderated these associations. Blood levels of IL-6 and vascular risk (e.g., homocysteine), as well as health history information, were collected. Processing speed assessments were administered to assess cognitive functioning, and we employed tract-based spatial statistics to examine whole brain white matter and regions of interest. Given the association between inflammation, vascular risk, and corpus callosum (CC) integrity, fractional anisotropy (FA) of the genu, body, and splenium represented our primary dependent variables. Whole brain analysis revealed an inverse association between IL-6 and CC fractional anisotropy. Subsequent ROI linear regression and ridge regression analyses indicated that the magnitude of this effect increased with age; thus, older individuals with higher IL-6 levels displayed lower white matter integrity. Finally, higher IL-6 levels were related to worse processing speed; this association was moderated by age, and was not fully accounted for by CC volume. This study highlights that at older ages, the association between higher IL-6 levels and lower white matter integrity is more pronounced; furthermore, it underscores the important, albeit burgeoning role of inflammatory processes in cognitive aging trajectories.