Primates can rapidly detect potential predators and modify their behavior based on the level of risk. The gaze direction of predators is one feature that primates can use to assess risk levels: recognition of a predators direct stare indicates to prey that it has been detected and the level of risk is relatively high. Predation has likely shaped visual attention in primates to quickly assess the level of risk but we know little about the constellation of low-level (e.g., contrast, color) and higher-order (e.g., category membership, perceived threat) visual features that primates use to do so. We therefore presented human and chimpanzee (Pan troglodytes) participants with photographs of potential predators (lions) and prey (impala) while we recorded their overt attention with an eye-tracker. The gaze of the predators and prey was either directed or averted. We found that both humans and chimpanzees visually fixated the eyes of predators more than those of prey. In addition, they directed the most attention toward the eyes of directed (rather than averted) predators. Humans, but not chimpanzees, gazed at the eyes of the predators and prey more than other features. Importantly, low-level visual features of the predators and prey did not provide a good explanation of the observed gaze patterns.