Brain plasticity is dynamically regulated across the life span, peaking during windows of early life. Typically assessed in the physiological range of milliseconds (real time), these trajectories are also influenced on the longer timescales of developmental time (nurture) and evolutionary time (nature), which shape neural architectures that support plasticity. Properly sequenced critical periods of circuit refinement build up complex cognitive functions, such as language, from more primary modalities. Here, we consider recent progress in the biological basis of critical periods as a unifying rubric for understanding plasticity across multiple timescales. Notably, the maturation of parvalbumin-positive (PV) inhibitory neurons is pivotal. These fast-spiking cells generate gamma oscillations associated with critical period plasticity, are sensitive to circadian gene manipulation, emerge at different rates across brain regions, acquire perineuronal nets with age, and may be influenced by epigenetic factors over generations. These features provide further novel insight into the impact of early adversity and neurodevelopmental risk factors for mental disorders.