How and when do infants gain the ability to learn entirely new information from language input alone? We propose that this ability requires that infants form a referential link between words, real-world objects or events, and mental representations of such objects or events. To investigate when this link emerges, we designed a novel paradigm that takes advantage of semantic priming and ostensive labeling. We showed that by 15 months, infants possess this referential link: they form representations of objects they have never seen and learn their names. Fourteen-moth-olds, however, failed on this task. This failure is surprising because prior research showed that infants at this age comprehend adults’ speech about hidden objects. New data from a follow-up study shows that bolstering younger infants’ vocabulary does not help them succeed on this task, suggesting that perhaps the referential link between words, referents and mental representations is fragile before 15 months.