We study human learning & decision-making in tasks with probabilistic rewards. Recent studies in a 2-armed bandit task find that a modification of classical Q-learning algorithms, with outcome-dependent learning rates, better explains behavior compared to constant learning rates. We propose a simple alternative: humans directly track the decision variable underlying choice in the task. Under this policy learning perspective, asymmetric learning can be reinterpreted as an increasing confidence in the preferred choice. We provide specific update rules for incorporating partial feedback (outcomes on chosen arms) and complete feedback (outcome on chosen & unchosen arms), and show that our model consistently outperforms previously proposed models on a range of datasets. Our model and update rules also add nuance to previous findings of perseverative behavior in bandit tasks; we show evidence of outcome-dependent choice perseveration, i.e., that humans persevere in their choices unless contradictory evidence is presented.

Simple perceptual decision-making tasks such as the Stroop and flanker tasks are popular as a method of measuring individual variation in the processing of conflicting visual stimuli--for instance, the difference in accuracy on stimuli with and without conflict. A major challenge in applying these tasks, for instance, to compare two different populations of subjects, is the low reliability of the nonparametric measures of performance in the tasks. Here, we model dynamic adjustments in decision policies often seen in human behavior, thereby capturing trial-by-trial variation in decision policies, in addition to the classically used average statistics. We propose a recurrent network model to capture behavioral strategies in the task in a model-agnostic manner, and to overcome small-sample learning challenges by pooling across subjects. We show that by splitting the learning into a complex, shared meta-model and simple subject-specific parameters, we learn significantly better predictive models, and also identify latent dimensions indexing the decision policy that may serve as a better measure of individual differences in the task.

Identification of neurocognitive predictors of substance dependence is an important step in developing approaches to prevent addiction. Given evidence of inhibitory control deficits in substance abusers (Monterosso et al., 2005; Fu et al., 2008; Lawrence et al., 2009; Tabibnia et al., 2011), we examined neural processing characteristics in human occasional stimulant users (OSU), a population at risk for dependence. A total of 158 nondependent OSU and 47 stimulant-naive control subjects (CS) were recruited and completed a stop signal task while undergoing functional magnetic resonance imaging (fMRI). A Bayesian ideal observer model was used to predict probabilistic expectations of inhibitory demand, P(stop), on a trial-to-trial basis, based on experienced trial history. Compared with CS, OSU showed attenuated neural activation related to P(stop) magnitude in several areas, including left prefrontal cortex and left caudate. OSU also showed reduced neural activation in the dorsal anterior cingulate cortex (dACC) and right insula in response to an unsigned Bayesian prediction error representing the discrepancy between stimulus outcome and the predicted probability of a stop trial. These results indicate that, despite minimal overt behavioral manifestations, OSU use fewer brain processing resources to predict and update the need for response inhibition, processes that are critical for adjusting and optimizing behavioral performance, which may provide a biomarker for the development of substance dependence.