A variety of lines of evidence support the idea that neutral evolutionary processes (genetic drift, mutation) have been important in generating cranial differences between Neandertals and modern humans. But how do Neandertals and modern humans compare with other species? And how do these comparisons illuminate the evolutionary processes underlying cranial diversification? To address these questions, we used 27 standard cranial measurements collected on 2524 recent modern humans, 20 Neandertals and 237 common chimpanzees to estimate split times between Neandertals and modern humans, and between Pan troglodytes verus and two other subspecies of common chimpanzee. Consistent with a neutral divergence, the Neandertal versus modern human split-time estimates based on cranial measurements are similar to those based on DNA sequences. By contrast, the common chimpanzee cranial estimates are much lower than DNA-sequence estimates. Apparently, cranial evolution has been unconstrained in Neandertals and modern humans compared with common chimpanzees. Based on these and additional analyses, it appears that cranial differentiation in common chimpanzees has been restricted by stabilizing natural selection. Alternatively, this restriction could be due to genetic and/or developmental constraints on the amount of within-group variance (relative to effective population size) available for genetic drift to act on.