New interaction potentials were developed for molecular dynamics simulations to study the role of Mg and Ca in modifying the structure and properties of alkaline earth silicates and borates. Competition between the depolymerization of the silica network and the formation of new bonds between oxygen atoms and modifiers leads to the enhancement of the elastic moduli with increasing modifier content in alkaline earth silicate glasses. Compared with calcium silicate, the higher elastic moduli of magnesium silicate result from a higher connectivity of the overall glass network due to the incorporation of fourfold coordinated magnesium and a more rigid connection between oxygen atoms and modifiers. In contrast to the silicates, the effect of modifier on the elastic moduli of alkaline earth borates is dominated by the formation of fourfold coordinated boron (N4). Calcium borate with higher N4 shows a more rigid network structure and higher elastic moduli.