Gray matter (GM) atrophy and brain glucose hypometabolism are already detected in the predementia stages of Alzheimer's disease (AD), but the regional and longitudinal associations between the two are not well understood. Here, we analyzed the patterns of longitudinal atrophy (magnetic resonance imaging [MRI]) and (18)F-Fluorodeoxyglucose-positron emission tomography ([18F]FDG-PET) metabolism decline in 40 cognitively healthy control (HC) and 52 mildly impaired (mild cognitive impairment [MCI]) subjects during 3 years. Based on cerebrospinal fluid and brain amyloid-PET, the subjects were divided into amyloid-beta (Aβ)- and Aβ+ subgroups. In voxel-based and region of interest analyses, we compared the 3-year rates of change in GM and glucose metabolism between Aβ-subgroups, within each diagnostic group. In joint-independent component analyses, we assessed the patterns of covariation between longitudinal change in GM volume and glucose metabolism. MCI-Aβ+ showed faster atrophy than MCI-Aβ- within the temporal, medial temporal, and medial parietal lobes. HC-Aβ+ showed faster atrophy within the precuneus than HC-Aβ-. For FDG-PET metabolism, MCI-Aβ+ exhibited faster decline than MCI-Aβ- in temporoparietal regions, whereas no differences between HC subgroups were observed. Joint-independent component analysis showed that accelerated atrophy and metabolism decline correlated across distant brain regions for MCI-Aβ+. In conclusion, abnormally increased levels of Aβ in nondemented subjects were associated with accelerated decline in both GM and glucose metabolism, where both types of neurodegeneration progress in spatially divergent patterns.