Aspergillus fumigatus is the most common cause of invasive mold disease in humans. The mechanisms underlying the adherence of this mold to host cells and macromolecules have remained elusive. Using mutants with different adhesive properties and comparative transcriptomics, we discovered that the gene uge3, encoding a fungal epimerase, is required for adherence through mediating the synthesis of galactosaminogalactan. Galactosaminogalactan functions as the dominant adhesin of A. fumigatus and mediates adherence to plastic, fibronectin, and epithelial cells. In addition, galactosaminogalactan suppresses host inflammatory responses in vitro and in vivo, in part through masking cell wall β-glucans from recognition by dectin-1. Finally, galactosaminogalactan is essential for full virulence in two murine models of invasive aspergillosis. Collectively these data establish a role for galactosaminogalactan as a pivotal bifunctional virulence factor in the pathogenesis of invasive aspergillosis.