Failing to come up with a word or name is a fairly common experience that is exacerbated in older adulthood and among populations with language impairments, and yet the mechanisms underlying lexical retrieval remain fairly understudied. In this work, we introduce and evaluate a series of nested computational models of lexical retrieval that combine semantic representations derived from a distributional semantic model with a process model to account for behavioral performance in a primed lexical retrieval task. The models were tested on a behavioral data set where participants attempted to retrieve answers to descriptions of low-frequency words and were provided a semantically and/or phonologically related prime word before the retrieval attempt. Model comparisons indicated that a model that emphasized semantic activations from the description and phonological activations from the prime word best accounted for the overall data. Additionally, incorrect responses and metacognitive judgments indicating that participants had other words in mind were associated with models that instead emphasized semantic activations from the prime word. Taken together, these results identify the locus of lexical retrieval failures and offer the opportunity to investigate broader questions about semantic memory retrieval.