Proceedings of the Annual Meeting of the Cognitive Science Society

Flexibly adapting behavior to different contexts is a critical component of human intelligence. It requires knowledge to be structured as coherent, context-dependent action rules, or task-sets (TS). Nevertheless, inferring optimal TS is compu- tationally complex. This paper tests the key predictions of a neurally-inspired model that employs hierarchically-structured reinforcement learning (RL) to approximate optimal inference. The model proposes that RL acts at two levels of abstrac- tion: a high-level RL process learns context-TS values, which guide TS selection based on context; a low-level process learns stimulus-actions values within TS, which guide action selec- tion in response to stimuli. In our novel task paradigm, we found evidence that participants indeed learned values at both levels: not only stimulus-action values, but also context-TS values affected learning and TS reactivation, and TS values alone determined TS generalization. This supports the claim of two RL processes, and their importance in structuring our interactions with the world.