Longitudinal brain image analysis is critical for revealing subtle but complex structural and functional changes of brain during aging or in neurodevelopmental disease. However, even with the rapid increase of clinical research and trials, a software toolbox dedicated for longitudinal image analysis is still lacking publicly. To cater for this increasing need, we have developed a dedicated 4D Adult Brain Extraction and Analysis Toolbox (aBEAT) to provide robust and accurate analysis of the longitudinal adult brain MR images. Specially, a group of image processing tools were integrated into aBEAT, including 4D brain extraction, 4D tissue segmentation, and 4D brain labeling. First, a 4D deformable-surface-based brain extraction algorithm, which can deform serial brain surfaces simultaneously under temporal smoothness constraint, was developed for consistent brain extraction. Second, a level-sets-based 4D tissue segmentation algorithm that incorporates local intensity distribution, spatial cortical-thickness constraint, and temporal cortical-thickness consistency was also included in aBEAT for consistent brain tissue segmentation. Third, a longitudinal groupwise image registration framework was further integrated into aBEAT for consistent ROI labeling by simultaneously warping a pre-labeled brain atlas to the longitudinal brain images. The performance of aBEAT has been extensively evaluated on a large number of longitudinal MR T1 images which include normal and dementia subjects, achieving very promising results. A Linux-based standalone package of aBEAT is now freely available at http://www.nitrc.org/projects/abeat.