Alzheimer's disease (AD) is a progressive and fatal neurodegenerative disease. The strongest genetic risk factor for sporadic AD is carriage of the ε4 allele of the Apolipoprotein E (APOE) gene. Strategies to slow the progression of AD, including dietary interventions, may be modified by the pathogenic effect of this polymorphism. Our objective in this review was to determine the extent and quality of the literature investigating how dietary factors and interventions interact with the APOE ε4 genotype to impact cognitive decline in AD. To that end, we performed a systematic scoping review of published English-language articles involving human subjects. We found evidence suggesting that adherence to a Mediterranean diet may reduce cognitive decline among APOE ε4 carriers, whereas ketogenic agents appear to be ineffective. Diets high in saturated fats may be particularly harmful for APOE ε4 carriers. We identified several topics, including the use of ω-3 fatty acid and antioxidant supplements, for which additional high level evidence is needed.