Proceedings of the Annual Meeting of the Cognitive Science Society

Animals exhibit behavioural flexibility by making reward-optimal decisions in unfamiliar situations. Computing value from minimal reward feedback requires leveraging known structure in the world. The Prefrontal Cortex plays a pivotal role in both value computation and learning world structure, however mechanistic explanations for each---within the frameworks of Reinforcement Learning and Cognitive Maps---are largely distinct. We develop a mechanistic model of PFC that unifies value and structure, in which value acts as a control signal that gates structured working memory representations to ultimately control behaviour. This is a significant departure from the conventional perspective of value as an action-readout. Critically, this model generalises to novel contexts without need for additional synaptic plasticity. We demonstrate that recurrent neural networks (RNNs) meta-trained with RL recapitulate this exact model on various value-based tasks. These results provide a parsimonious explanation of both the value and schema frameworks of PFC function.