Proceedings of the Annual Meeting of the Cognitive Science Society

When faced with delayed, uncertain rewards, humans andother animals usually prefer to know the eventual outcomesin advance. This preference for cues providing advance infor-mation can lead to seemingly suboptimal choices, where lessreward is preferred over more reward. Here, we introduce areinforcement-learning model of this behavior, the anticipatedprediction error (APE) model, based on the idea that predic-tion errors themselves can be rewarding. As a result, animalswill sometimes pick options that yield large prediction errors,even when the expected rewards are smaller. We compare theAPE model against an alternative information-bonus model,where information itself is viewed as rewarding. These mod-els are evaluated against a newly collected dataset with humanparticipants. The APE model fits the data as well or betterthan the other models, with fewer free parameters, thus provid-ing a more robust and parsimonious account of the suboptimalchoices. These results suggest that anticipated prediction er-rors can be an important signal underpinning decision making.