- Main
Environmental Statistics Shape Learning Dynamics in Meta-Reinforcement Learning
Abstract
Adverse environments have been proposed to influence multiple aspects of cognition, including learning and adaptation. However, causal tests of long-term effects of adversity on human cognition are extremely difficult. Moreover, different types of adversity often co-occur in real life, making it challenging to disentangle their individual effects. Computational models provide a way to independently manipulate specific environmental statistics and examine their causal impact on learning. Here, we integrate meta-learning with reinforcement learning to investigate the mechanisms by which environmental statistics shape learning processes. Specifically, we examine how two core environmental dimensions of adversity, volatility and controllability, shape trajectories of learning rates in deep neural networks. We find that both high volatility and low controllability reduce the maximum learning rate, but only controllability affects the initial slope of learning, leading to a flattening of early learning dynamics. Notably, deeper layers of the model are more strongly influenced by environmental factors than earlier layers, leading to reduced hierarchical differentiation in learning rates under conditions of high volatility and low controllability. These results suggest that a meta–reinforcement learning framework may provide a simplified yet useful approach for gaining mechanistic insight into how distinct types of adverse environments affect learning.