Cerebral atherosclerosis (AS), small vessel disease (SVD), and cerebral amyloid angiopathy (CAA) are the most prevalent arterial disorders in the aged brain. Pathogenetically, AS and SVD share similar mechanisms: plasma protein leakage into the vessel wall, accumulation of lipid-containing macrophages, and fibrosis of the vessel wall. CAA, on the other hand, is characterized by the deposition of the amyloid β-protein in the vessel wall. Despite these differences between CAA, AS and SVD, apolipoprotein E (apoE) is involved in all three disorders. Such a pathogenetic link may explain the correlations between AS, SVD, CAA, and Alzheimer’s disease in the brains of elderly individuals reported in the literature. In addition, AS, SVD, and CAA can lead to tissue lesions such as hemorrhage and infarction. Moreover, intracerebral SVD leads to plasma protein leakage into the damaged vessel wall and into the perivascular space resulting in a blood–brain barrier (BBB) dysfunction. This SVD-related BBB dysfunction is considered to cause white matter lesions (WMLs) and lacunar infarcts. In this review, we demonstrate the relationship between AS, SVD, and CAA as well as their contribution to the development of vascular tissue lesions and we emphasize an important role for apoE in the pathogenesis of vessel disorders and vascular tissue lesions as well as for BBB dysfunction on WML and lacunar infarct development.