Visual working memory and visual mental imagery both involve the use of internal visual representations, and they likely have overlapping neural substrates. However, research on people with “aphantasia,” or a lack of visual imagery, has not found any evidence that aphantasics are impaired on visual working memory tasks, possibly because they can use non-visual strategies. We designed a task intended to prevent compensatory strategies, and also to explore what happens when aphantasics are required to shift the focus of attention between items in working memory. We found that aphantasics were not significantly different from controls, either when maintaining or shifting the focus of attention. Explanations include non-visual memory strategies, but also the possibility that aphantasics can store information in visual working memory without conscious awareness. Future research should combine behavioral methods with neuroimaging to investigate how aphantasics encode working memory representations.