Anxiety disorders are a major public health concern and current treatments are inadequate for many individuals. Anxiety is more common in women than men and this difference arises during puberty. Sex differences in physiological stress responses may contribute to this variability. During puberty, gonadal hormones shape brain structure and function, but the extent to which these changes affect stress sensitivity is unknown. We examined how pubertal androgens shape behavioral and neural responses to social stress in California mice (Peromyscus californicus), a model species for studying sex differences in stress responses. In adults, social defeat reduces social approach and increases social vigilance in females but not males. We show this sex difference is absent in juveniles, and that prepubertal castration sensitizes adult males to social defeat. Adult gonadectomy does not alter behavioral responses to defeat, indicating that gonadal hormones act during puberty to program behavioral responses to stress in adulthood. Calcium imaging in the medioventral bed nucleus of the stria terminalis (BNST) showed that social threats increased neural activity and that prepubertal castration generalized these responses to less threatening social contexts. These results support recent hypotheses that the BNST responds to immediate threats. Prepubertal treatment with the nonaromatizable androgen dihydrotestosterone acts in males and females to reduce the effects of defeat on social approach and vigilance in adults. These data indicate that activation of androgen receptors during puberty is critical for programming behavioral responses to stress in adulthood.