Proceedings of the Annual Meeting of the Cognitive Science Society

Human moral judgments are both precise, with clear intuitions about right and wrong, and at the same time obscure, as they seem to result from principles whose logic often escapes us. The development of Artificial Intelligence (AI) applications requires an understanding of this subtle logic if we are to embed moral considerations in artificial systems. Reinforcement Learning (RL) algorithms have emerged as a valuable interactive tool for investigating moral behavior. However, being value-based algorithms, they face difficulty when it comes to explaining deontological, non-consequentialist moral judgments. Here, in a multi-agent learning scenario based on the Producer-Scrounger Game, we show that RL agents can converge towards apparently non-consequentialist outcomes, provided the algorithm accounts for the temporal value of actions. The implications of our findings extend to integrating morality into AI agents by elucidating the interplay between learning strategies, characteristics for accounting temporal values, and methods of considering the opponent's payoff.