Terrestrial primary productivity and carbon cycle impacts of droughts are commonly quantified using vapour pressure deficit (VPD) data and remotely sensed greenness, without accounting for soil moisture. However, soil moisture limitation is known to strongly affect plant physiology. Here, we investigate light use efficiency, the ratio of gross primary productivity (GPP) to absorbed light. We derive its fractional reduction due to soil moisture (fLUE), separated from VPD and greenness changes, using artificial neural networks trained on eddy covariance data, multiple soil moisture datasets and remotely sensed greenness. This reveals substantial impacts of soil moisture alone that reduce GPP by up to 40% at sites located in sub-humid, semi-arid or arid regions. For sites in relatively moist climates, we find, paradoxically, a muted fLUE response to drying soil, but reduced fLUE under wet conditions. fLUE identifies substantial drought impacts that are not captured when relying solely on VPD and greenness changes and, when seasonally recurring, are missed by traditional, anomaly-based drought indices. Counter to common assumptions, fLUE reductions are largest in drought-deciduous vegetation, including grasslands. Our results highlight the necessity to account for soil moisture limitation in terrestrial primary productivity data products, especially for drought-related assessments.