Measuring early tau accumulation is important in studying aging and Alzheimer disease and is only as accurate as the signal-to-noise ratio of the tracer. Along with aggregated tau in the form of neurofibrillary tangles, F-flortaucipir has been reported to bind to neuromelanin, monoamine oxidase, calcifications, iron, leptomeningeal melanocytes, and microhemorrages. Although F-flortaucipir successfully differentiates healthy controls (HCs) from subjects with Alzheimer disease, variability exists in the cortical signal in amyloid-negative HCs. We aimed to explore the relationship between off-target binding signal and variability in the cortical signal in HCs. Subjects ( = 139) received C-Pittsburgh compound B (PIB) and F-flortaucipir PET scans and a magnetization-prepared rapid gradient echo MRI scan. PET frames were realigned and coregistered to the MR images, which were segmented using FreeSurfer. In amyloid-negative HCs ( = 90; age range, 21-94 y), 7 nonspecific or off-target binding regions were considered: caudate, pallidum, putamen, thalamus, cerebellar white matter, hemispheric white matter, and choroid plexus. These regions of interest were assigned to 3 similarly behaving groups using principle components analysis, exploratory factor analysis, and Pearson correlations for caudate, putamen, and pallidum (also correlated with age); thalamus and white matter; and choroid plexus. In amyloid-negative HCs with C-PIB and F-flortaucipir scans, correlations were calculated between white and gray matter before and after partial-volume correction. The correlation between white and gray matter disappeared after partial-volume correction in C-PIB ( = 0) but persisted for F-flortaucipir ( = 0.27), demonstrating that the correlation between white and gray matter signal in F-flortaucipir is not solely due to partial-volume effects. A linear regression showed that off-target signal from putamen and thalamus together explained 64% of the variability in the cortical signal in amyloid-negative HCs (not seen in amyloid-positive HCs). Variability in amyloid-negative HCs but not amyloid-positive HCs correlated with white matter signal (unrelated to partial-volume effects) and age-related off-target signal (possibly related to iron load). The noise in the F-flortaucipir measurement could pose challenges when studying early tau accumulation.