Proceedings of the Annual Meeting of the Cognitive Science Society

Human decision-making in dynamic environments exhibits substantial trial-by-trial variability that fixed-parameter reinforcement learning (RL) models fail to capture. Recent data-driven approaches infer time-varying RL parameters from behavior, but their dynamics remain purely descriptive and lack normative grounding. We propose Normative DynamicRL, a framework that integrates dynamic parameter estimation with explicit normative constraints derived from regret minimization. Instead of allowing arbitrary parameter fluctuations, our approach regularizes learning rates, exploration temperatures, and perseveration toward values that reduce expected regret given environmental volatility, stochasticity, and horizon. This formulation bridges descriptive behavioral modeling and normative theories of adaptive control. Across multiple non-stationary decision-making tasks, Normative DynamicRL improves generalization to unseen environmental regimes while maintaining predictive performance and interpretability, enabling principled assessment of when and how human strategy adaptation aligns with rational optimality. Our results further suggest that incorporating normative structure provides a robust inductive bias for learning adaptive strategies under uncertainty.