Proper regulation of organelle dynamics and inter-organelle contacts is critical for cellular health and function. Both the endoplasmic reticulum (ER) and actin cytoskeleton are known to regulate organelle dynamics, but how, when, and where these two subcellular components are coordinated to control organelle dynamics remains unclear. Here, we show that ER-associated actin consistently marks mitochondrial, endosomal, and lysosomal fission sites. We also show that actin polymerization by the ER-anchored isoform of the formin protein INF2 is a key regulator of the morphology and mobility of these organelles. Together, our findings establish a mechanism by which INF2-mediated polymerization of ER-associated actin at ER-organelle contacts regulates organelle dynamics.