A computerized version of a popular children’s memory game (“Concentration”) was used to test the role of language in visuospatial working memory of humans, apes, and monkeys. Participants were required to find matching pairs of pictures by “flipping over” computer-generated cards, and to remember which images had been seen and where each was hidden. All participants were able to locate the pairs of stimuli, but the nonhuman animals were consistently and significantly worse than the human adults. When humans could not use language, performance declined. When the stimuli were meaningful symbols from the chimpanzees’ language keyboards, performance improved. These data suggest that language provides an important function even in visuospatial working memory, linking “memory for what” with “memory for where.”