Background: Among postmenopausal breast cancer survivors, poor physical health has been associated with higher risks of breast cancer events. Obesity and physical inactivity are known risk factors for poor physical health, while circulating estrogen is an additional potential risk factor. We tested the hypothesis that the relationship between poor physical health and worse breast cancer outcomes is mediated by higher estrogen concentrations associated with body size and physical inactivity. Methods: We used data from 1030 postmenopausal breast cancer survivors to examine the association between serum estradiol levels, body mass index (BMI), physical activity, and RAND-36-item Health Survey (SF-36) physical health. Results: In univariate analysis, poor physical health was associated with higher estradiol levels, in addition to obesity and low physical activity. Higher estradiol levels were significantly associated with higher odds of poor physical health (odds ratio, OR, 1.20 [95% confidence interval 1.03-1.39]) in a multivariable model adjusting for age, cancer stage and treatment, alcohol use, and physical activity. However, the relationship between estradiol levels and poor physical health was no longer significant (OR 1.06 [0.91-1.24]) after adding BMI in the model. In multivariate analysis, only poor physical health resulted in higher risks of recurrence (hazard ratio 1.33 [95% CI 1.08-1.64]). Conclusions: These findings indicate that estradiol is related to poor physical health, but is not an independent risk factor from body size or inactivity. While obesity and physical activity in survivorship are potential targets for improving physical health, other biological processes that impact physical health, e.g. inflammation, remain to be identified. © Mary Ann Liebert, Inc.