The medial temporal lobe (MTL) is an early site of tau accumulation and MTL dysfunction may underlie episodic-memory decline in aging and dementia. Postmortem data indicate that tau pathology in the transentorhinal cortex is common by age 60, whereas spread to neocortical regions and worsening of cognition is associated with β-amyloid (Aβ). We used [18F]AV-1451 and [11C]PiB positron emission tomography, structural MRI, and neuropsychological assessment to investigate how in vivo tau accumulation in temporal lobe regions, Aβ, and MTL atrophy contribute to episodic memory in cognitively normal older adults (n = 83; age, 77 ± 6 years; 58% female). Stepwise regressions identified tau in MTL regions known to be affected in old age as the best predictor of episodic-memory performance independent of Aβ status. There was no interactive effect of MTL tau with Aβ on memory. Higher MTL tau was related to higher age in the subjects without evidence of Aβ. Among temporal lobe subregions, episodic memory was most strongly related to tau-tracer uptake in the parahippocampal gyrus, particularly the posterior entorhinal cortex, which in our parcellation includes the transentorhinal cortex. In subjects with longitudinal MRI and cognitive data (n = 57), entorhinal atrophy mirrored patterns of tau pathology and their relationship with memory decline. Our data are consistent with neuropathological studies and further suggest that entorhinal tau pathology underlies memory decline in old age even without Aβ.SIGNIFICANCE STATEMENT Tau tangles and β-amyloid (Aβ) plaques are key lesions in Alzheimer's disease (AD) but both pathologies also occur in cognitively normal older people. Neuropathological data indicate that tau tangles in the medial temporal lobe (MTL) underlie episodic-memory impairments in AD dementia. However, it remains unclear whether MTL tau pathology also accounts for memory impairments often seen in elderly people and how Aβ affects this relationship. Using tau-specific and Aβ-specific positron emission tomography tracers, we show that in vivo MTL tau pathology is associated with episodic-memory performance and MTL atrophy in cognitively normal adults, independent of Aβ. Our data point to MTL tau pathology, particularly in the entorhinal cortex, as a substrate of age-related episodic-memory loss.