Although high mammographic density is considered one of the strongest risk factors for invasive breast cancer, the genes involved in modulating this clinical feature are unknown. Tissues of high mammographic density share key histologic features with stromal components within malignant lesions of tumor tissues, specifically low adipocyte and high extracellular matrix (ECM) content. We show that CD36, a transmembrane receptor that coordinately modulates multiple protumorigenic phenotypes, including adipocyte differentiation, angiogenesis, cell-ECM interactions, and immune signaling, is greatly repressed in multiple cell types of disease-free stroma associated with high mammographic density and tumor stroma. Using both in vitro and in vivo assays, we show that CD36 repression is necessary and sufficient to recapitulate the above-mentioned phenotypes observed in high mammographic density and tumor tissues. Consistent with a functional role for this coordinated program in tumorigenesis, we observe that clinical outcomes are strongly associated with CD36 expression.CD36 simultaneously controls adipocyte content and matrix accumulation and is coordinately repressed in multiple cell types within tumor and high mammographic density stroma, suggesting that activation of this stromal program is an early event in tumorigenesis. Levels of CD36 and extent of mammographic density are both modifiable factors that provide potential for intervention.