Substituting heteroatoms into graphene can tune its properties for applications ranging from catalysis to spintronics. The further recent discovery that covalent impurities in graphene can be manipulated at atomic precision using a focused electron beam may open avenues towards sub-nanometer device architectures. However, the preparation of clean samples with a high density of dopants is still very challenging. Here, we report vacancy-mediated substitution of aluminium into laser-cleaned graphene, and without removal from our ultra-high vacuum apparatus, study their dynamics under 60 keV electron irradiation using aberration-corrected scanning transmission electron microscopy and spectroscopy. Three- and four-coordinated Al sites are identified, showing excellent agreement with ab initio predictions including binding energies and electron energy-loss spectrum simulations. We show that the direct exchange of carbon and aluminium atoms predicted earlier occurs under electron irradiation, although unexpectedly it is less probable than the same process for silicon. We also observe a previously unknown nitrogen-aluminium exchange that occurs at Al─N double-dopant sites at graphene divacancies created by our plasma treatment.