Measuring early tau accumulation is important in studying aging and Alzheimer's disease (AD) and is only as accurate as the signal to noise ratio of the tracer. Along with aggregated tau in the form of neurofibrillary tangles, 18F-Flortaucipir (18F-FTP) has been reported to bind to neuromelanin, monoamine oxidase, calcification, iron, lepto-meningeal melanocytes, and microhemorrages. Although 18F-FTP successfully differentiates healthy controls (HC) from subjects with AD, variability exists in the cortical signal in amyloid-β negative (Aβ-) HCs. We aimed to explore the relationship between off-target binding signal and variability in the cortical signal in HCs. Subjects (n = 139) received a 11C-PIB and 18F-FTP-PET scan, and MRI MPRAGE. PET frames were realigned and coregistered to the MRI, and MRIs were segmented using Freesurfer. In HCAβ- subjects (n = 90, age range=21-94), 7 non-specific or off-target binding regions were considered: caudate, pallidum, putamen, thalamus, cerebellar white, hemispheric white, and choroid plexus. These ROIs were assigned to 3 similarly behaving groups using principle component analysis, exploratory factor analysis and Pearson correlations: 1. caudate, putamen and pallidum, (also correlated with age), 2. thalamus and white matter, 3. choroid plexus. In HCAβ- subjects with 11C-PIB and 18F-FTP scans, correlations were calculated between white and gray matter before and after partial volume correction (PVC). The correlation between white and gray matter disappeared after PVC in 11C-PIB (r2=0.19 to 0), but persisted for 18F-FTP (r2=0.84 to 0.27), demonstrating that the correlation between white and gray matter signal in 18F-FTP is not solely due to partial volume effects. A linear regression showed that off-target signal from putamen and thalamus together explained 64% of the variability in the cortical signal in HCAβ- (not seen in HCAβ+). Lastly, mean 18F-FTP Standard Uptake Value Ratio was highly correlated to reported iron load from a previously published study. Variability in HCAβ-, but not HCAβ+, correlated with an age-related off-target signal, which could be related to iron load as well as white matter signal. The noise in the 18F-FTP measurement could pose challenges when studying early tau accumulation.