The cosmic accretion of both dark matter and baryons into halos is typically measured using some evolving virial relation, but recent work suggests that most halo growth at late cosmic time (z ≲ 2) is not physical but is rather the by-product of an evolving virial radius ("pseudo-evolution"). Using Omega25, a suite of cosmological simulations that incorporate both dark matter and gas dynamics with differing treatments of gas cooling, star formation, and thermal feedback, we systematically explore the physics that governs cosmic accretion into halos and their galaxies. Physically meaningful cosmic accretion of both dark matter and baryons occurs at z ≳ 1 across our halo mass range: M200m 10 = 11-14 M⊙. However, dark matter, because it is dissipationless, is deposited (in a timeaverage sense) at ≳R200m (z) in a shell-like manner, such that dark matter mass and density experience little-to-no physical growth at any radius within a halo at z < 1. In contrast, gas, because it is able to cool radiatively, experiences significant accretion at all radii, at a rate that roughly tracks the accretion rate at R200m, at all redshifts. Infalling gas starts to decouple from dark matter at ≈2 R200m and continues to accrete to smaller radii until the onset of strong angular-momentum support at ≈0.1 R200m. Thus, while the growth of dark matter is subject to pseudoevolution, the growth of baryons is not. The fact that the accretion rate of gas on galactic scales tracks the accretion rate near R200m provides insight into the tight relations between the masses/sizes of galaxies and those of their host halos across cosmic time.