We present a unified theoretical framework for computing spin-independent direct detection rates via various channels relevant for sub-GeV dark matter — nuclear re- coils, electron transitions and single phonon excitations. Despite the very different physics involved, in each case the rate factorizes into the particle-level matrix element squared, and an integral over a target material- and channel-specific dynamic structure factor. We show how the dynamic structure factor can be derived in all three cases following the same procedure, and extend previous results in the literature in several aspects. For electron transitions, we incorporate directional dependence and point out anisotropic target materials with strong daily modulation in the scattering rate. For single phonon excitations, we present a new derivation of the rate formula from first principles for generic spin-independent couplings, and include the first calculation of phonon excitation through electron couplings. We also discuss the interplay between single phonon excitations and nuclear recoils, and clarify the role of Umklapp processes, which can dominate the single phonon production rate for dark matter heavier than an MeV. Our results highlight the complementarity between various search channels in probing different kinematic regimes of dark matter scattering, and provide a common reference to connect dark matter theories with ongoing and future direct detection experiments.