We studied the relationship of prenatal and post-birth exposure to particulate matter < 2.5 μm in diameter (PM2.5) with infant mortality for all births between 2011 and 2013 in the conterminous United States. Prenatal exposure was defined separately for each trimester, post-birth exposure was defined in the 12 months following the prenatal period, and infant mortality was defined as death in the first year of life. For the analysis, we merged over 10 million cohort-linked live birth-infant death records with daily, county-level PM2.5 concentration data and then fit a Structural Equation Model controlling for several individual- and county-level confounders. We estimated direct paths from the two exposures to infant death as well as indirect paths from the prenatal exposure to the outcome through preterm birth and low birth weight. Prenatal PM2.5 exposure was positively associated with infant death across all trimesters, although the relationship was strongest in the third trimester. The direct pathway from the prenatal exposure to the outcome accounted for most of this association. Estimates for the post-birth PM2.5-infant death relationship were less precise. The results from our study add to a growing literature that provides evidence in favor of the potential harmful effects on human health of low levels of air pollution.