Levels of circulating of sex hormones are associated with glucose metabolism and adiposity, but little is known about their association with ectopic fat. We aimed to characterize the association between circulating sex hormones and liver fat. Methods: We conducted a cross-sectional analysis by using data from the Multiethnic Study of Atherosclerosis to assess the association of the circulating levels of bioavailable testosterone, estradiol, dehydroepiandrosterone, and sex hormone binding globulin (SHBG) with fatty liver. Fatty liver was defined as a reduction of ≤40 Hounsfield units, measured by computed tomography, in 2835 postmenopausal women and 2899 men (45-84 years old; white, black, Hispanic, or Chinese) at 6 centers in the United States. Results: Women in the highest tertile of bioavailable testosterone were significantly more likely to have fatty liver than women in the lowest tertile (odds ratio, 1.77; 95% confidence interval, 1.07-2.92). We found an even greater difference for level of estradiol (odds ratio, 2.49; 95% confidence interval, 1.41-4.39) after adjusting for age, race/ethnicity, waist-to-hip ratio, hypertension, total and high-density lipoprotein cholesterol, smoking, insulin sensitivity, and hormone replacement therapy use. Men in the highest tertile of estradiol level were significantly more likely to have fatty liver than men in the lowest tertile (odds ratio, 2.10; 95% confidence level, 1.29-3.40). Men in the highest tertile of SHBG were less likely to have fatty liver than those in the lowest tertile (odds ratio, 0.46; 95% confidence interval, 0.27-0.77). Other associations between hormone levels and fatty liver were not statistically significant. Conclusions: On the basis of a cross-sectional study, postmenopausal women with high levels of bioavailable testosterone are at greater risk for fatty liver. In men, higher levels of SHBG are associated with reduced risk for fatty liver. Higher levels of estradiol are associated with fatty liver in both sexes. This pattern is consistent with the sex-specific associations of sex hormones with other cardiometabolic risk factors.