There is evidence that differences among individuals in white matter microstructure, as measured with diffusion tensor imaging (DTI), are under genetic control. However, little is known about the relative contribution of genetic and environmental effects on different diffusivity indices among late middle-aged adults. Here, we examined the magnitude of genetic influences for fractional anisotropy (FA), and mean (MD), axial (AD), and radial (RD) diffusivities in male twins aged 56-66 years old. Using an atlas-based registration approach to delineate individual white matter tracts, we investigated mean DTI-based indices within the corpus callosum, 12 bilateral tracts and all these regions of interest combined. All four diffusivity indices had high heritability at the global level (72%-80%). The magnitude of genetic effects in individual tracts varied from 0% to 82% for FA, 0% to 81% for MD, 8% to 77% for AD, and 0% to 80% for RD with most of the tracts showing significant heritability estimates. Despite the narrow age range of this community-based sample, age was correlated with all four diffusivity indices at the global level. In sum, all diffusion indices proved to have substantial heritability for most of the tracts and the heritability estimates were similar in magnitude for different diffusivity measures. Future studies could aim to discover the particular set of genes that underlie the significant heritability of white matter microstructure. Hum Brain Mapp 38:2026-2036, 2017. © 2017 Wiley Periodicals, Inc.