The diffusive propagation of magnons in the archetypal magnetic insulator yttrium iron garnet (YIG) is being actively explored for low-power and low-loss data communication. However, operation under external magnetic fields reduces the magnon diffusion length and attenuates the voltage amplitude at measurement terminals of magnonic devices. Here, we explore the low-field and field-free regime of diffusive magnon transport in YIG films, demonstrating that the field-induced suppression of magnon diffusion length can be fully inhibited only at the zero-field limit. Even a modest field of 10 mT attenuates the nonlocal spin voltage by ∼20% in an ∼1 μm long transport channel. We further identify the often overlooked in-plane uniaxial magnetic anisotropy as the dominant factor governing magnon transport in the low-field regime. Using Stoner-Wohlfarth macrospin simulations, we quantify the anisotropy parameters and reveal a 10-fold enhancement at low temperatures, a key finding for field-free operation of magnonic devices under cryogenic conditions.