Neuroendocrine carcinomas (NEC) of the breast are exceedingly rare tumors, which are classified in the WHO system as small cell (SCNEC) and large cell (LCNEC) carcinoma based on indistinguishable features from their lung counterparts. In contrast to lung and enteropancreatic NEC, the genomics of breast NEC have not been well-characterized. In this study, we examined the clinicopathologic, immunohistochemical, and genetic features of 13 breast NEC (7 SCNEC, 4 LCNEC, 2 NEC with ambiguous small versus large cell morphology [ANEC]). Co-alterations of TP53 and RB1 were identified in 86% (6/7) SCNEC, 100% (2/2) ANEC, and 50% (2/4) LCNEC. The one SCNEC without TP53/RB1 alteration had other p53 pathway aberrations (MDM2 and MDM4 amplification) and was immunohistochemically RB negative. PIK3CA/PTEN pathway alterations and ZNF703 amplifications were each identified in 46% (6/13) NEC. Two tumors (1 SCNEC, 1 LCNEC) were CDH1 mutated. By immunohistochemistry, 100% SCNEC (6/6) and ANEC (2/2) and 50% (2/4) LCNEC (83% NEC) showed RB loss, compared to 0% (0/8) grade 3 neuroendocrine tumors (NET) (p < 0.001) and 38% (36/95) grade 3 invasive ductal carcinomas of no special type (IDC-NST) (p = 0.004). NEC were also more often p53 aberrant (60% vs 0%, p = 0.013), ER negative (69% vs 0%, p = 0.005), and GATA3 negative (67% vs 0%, p = 0.013) than grade 3 NET. Two mixed NEC had IDC-NST components, and 69% (9/13) of tumors were associated with carcinoma in situ (6 neuroendocrine DCIS, 2 non-neuroendocrine DCIS, 1 non-neuroendocrine LCIS). NEC and IDC-NST components of mixed tumors were clonally related and immunophenotypically distinct, lacking ER and GATA3 expression in NEC relative to IDC-NST, with RB loss only in NEC of one ANEC. The findings provide insight into the pathogenesis of breast NEC, underscore their classification as a distinct tumor type, and highlight genetic similarities to extramammary NEC, including highly prevalent p53/RB pathway aberrations in SCNEC.