White matter disease in the brain increases with age and cardiovascular disease, emerging in midlife, and these associations may be influenced by both genetic and environmental factors. We examined the frequency, distribution, and heritability of abnormal white matter and its association with hypertension in 395 middle-aged male twins (61.9 ± 2.6 years) from the Vietnam Era Twin Study of Aging, 67% of whom were hypertensive. A multi-channel segmentation approach estimated abnormal regions within the white matter. Using multivariable regression models, we characterized the frequency distribution of abnormal white matter in midlife and investigated associations with hypertension and Apolipoprotein E-ε4 status and the impact of duration and control of hypertension. Then, using the classical twin design, we estimated abnormal white matter heritability and the extent of shared genetic overlap with blood pressure. Abnormal white matter was predominantly located in periventricular and deep parietal and frontal regions; associated with age (t = 1.9, p = 0.05) and hypertension (t = 2.9, p = 0.004), but not Apolipoprotein ε4 status; and was greater in those with uncontrolled hypertension relative to controlled (t = 3.0, p = 0.003) and normotensive (t = 4.0, p = 0.0001) groups, suggesting that abnormal white matter may reflect currently active cerebrovascular effects. Abnormal white matter was highly heritable (a2 = 0.81) and shared some genetic influences with systolic blood pressure (rA = 0.26), although there was evidence for distinct genetic contributions and unique environmental influences. Future longitudinal research will shed light on factors impacting white matter disease presentation, progression, and potential recovery.