Cancer immunotherapy has been heralded as a breakthrough cancer treatment demonstrating tremendous success in improving tumor responses and survival of patients with hematological cancers and solid tumors. This novel promising treatment approach has in particular triggered optimism for triple negative breast cancer (TNBC) treatment, a subtype of breast cancer with distinct clinical features and poor clinical outcome. In early 2019, the FDA granted the first approval of immune checkpoint therapy, targeting PD-L1 (Atezolizumab) in combination with chemotherapy for the treatment of patients with locally advanced or metastatic PD-L1 positive TNBC. The efficacy of immuno-based interventions varies across cancer types and patient cohorts, which is attributed to a variety of lifestyle, clinical, and pathological factors. For instance, obesity has emerged as a risk factor for a dampened anti-tumor immune response and increased risk of immunotherapy-induced immune-related adverse events (irAEs) but has also been linked to improved outcomes with checkpoint blockade. Given the breadth of the rising global obesity epidemic, it is imperative to gain insight into the immunomodulatory effects of obesity in the peripheral circulation and within the tumor microenvironment. In this review, we resolve the impact of obesity on breast tumorigenesis and progression on the one hand, and on the immune contexture on the other hand. Finally, we speculate on the potential implications of obesity on immunotherapy response in breast cancer. This review clearly highlights the need for in vivo obese cancer models and representative clinical cohorts for evaluation of immunotherapy efficacy.