Developmental dyslexia is a common reading disorder that negatively impacts an individual's ability to achieve literacy. Although the brain network involved in reading and its dysfunction in dyslexia has been well studied, it is unknown whether dyslexia is caused by structural abnormalities in the reading network itself or in the lower-level networks that provide input to the reading network. In this study, we acquired structural magnetic resonance imaging scans longitudinally from 27 Norwegian children from before formal literacy training began until after dyslexia was diagnosed. Thus, we were able to determine that the primary neuroanatomical abnormalities that precede dyslexia are not in the reading network itself, but rather in lower-level areas responsible for auditory and visual processing and core executive functions. Abnormalities in the reading network itself were only observed at age 11, after children had learned how to read. The findings suggest that abnormalities in the reading network are the consequence of having different reading experiences, rather than dyslexia per se, whereas the neuroanatomical precursors are predominantly in primary sensory cortices.