Behavior is shaped by genes, environment, and evolutionary history in different ways. Nest architecture is an extended phenotype that results from the interaction between the behavior of animals and their environment. These extended phenotypes differ in structure among species and among colonies within a species, but the source of these differences remains an open question. To investigate the impact of colony identity (genetics), evolutionary history (species), and the environment on nest architecture, we compared how two species of harvester ants, Pogonomyrmex californicus and Veromessor andrei, construct their nests under different environmental conditions. For each species, we allowed workers from four colonies to excavate nests in environments that differed in temperature and humidity for seven days. We then created casts of each nest to compare nest structures among colonies, between species, and across environmental conditions. We found differences in nest structure among colonies of the same species and between species. Interestingly, however, environmental conditions did not have a strong influence on nest structure in either species. Our results suggest that extended phenotypes are shaped more strongly by internal factors, such as genes and evolutionary history, and are less plastic in response to the abiotic environment, like many physical and physiological phenotypes.