A decline in mitochondrial function is a hallmark of aging and neurodegenerative diseases. It has been proposed that changes in mitochondrial morphology, including fragmentation of the tubular mitochondrial network, can lead to mitochondrial dysfunction, yet the mechanism of this loss of function is unclear. Most proteins contained within mitochondria are nuclear-encoded and must be properly targeted to the mitochondria. Here, we report that sustained mRNA localization and co-translational protein delivery leads to a heterogeneous protein distribution across fragmented mitochondria. We find that age-induced mitochondrial fragmentation drives a substantial increase in protein expression noise across fragments. Using a translational kinetic and molecular diffusion model, we find that protein expression noise is explained by the nature of stochastic compartmentalization and that co-translational protein delivery is the main contributor to increased heterogeneity. We observed that cells primarily reduce the variability in protein distribution by utilizing mitochondrial fission-fusion processes rather than relying on the mitophagy pathway. Furthermore, we are able to reduce the heterogeneity of the protein distribution by inhibiting co-translational protein targeting. This research lays the framework for a better understanding of the detrimental impact of mitochondrial fragmentation on the physiology of cells in aging and disease.