Previous work proposes that heuristics, such as Take-The-Best, may succeed because of deliberate ignorance of covari-ance in their cue weight estimates as opposed to full-information models (logistic regression). Other studies find thatTake-The-Best performs particularly well compared to full-information models in high covariance as opposed to low co-variance environments. This poses the question of whether heuristics perform well when there is a mismatch between theircovariance prior and the covariance in the environment? We test this by gradually manipulating solely the level of covari-ance among cues. Indeed, Take-The-Best performs better as average covariance increases, while tallying, nave Bayes andlogistic regression worsen. Since both nave Bayes and tallying also disregard covariance but integrate across cues, thisindicates the competitive advantage of Take-The-Best stems from relying on a single cue when redundancy is high. Weextend previous work by Rieskamp and Dieckmann (2012) and imply a reinterpretation of past Take-The-Bests successes.

To what extent do human reward learning and decision-making rely on the ability to represent and generate richly structured relationships between options? We provide evidence that structure learning and the principle of compositionality play crucial roles in human reinforcement learning. In a new multi-armed bandit paradigm, we found evidence that participants are able to learn representations of different reward structures and combine them to make correct generalizations about options in novel contexts. Moreover, we found substantial evidence that participants transferred knowledge of simpler reward structures to make compositional generalizations about rewards in complex contexts. This allowed participants to accumulate more rewards earlier, and to explore less whenever such knowledge transfer was possible. We also provide a computational model which is able to generalize and compose knowledge for complex reward structures. This model describes participant behaviour in the compositional generalization task better than various other models of decision-making and transfer learning.

Human reinforcement learning (RL) is characterized by different challenges. Exploration has been studied extensively in multi-armed bandits, while planning has been investigated in multi-step decision tasks. More recent work has added structured rewards to study generalization. However, past studies have often focused on a single one of these aspects, making it hard to compare results. We propose a generative model for constructing correlated trees to provide a unified and scalable method for studying exploration, planning, and generalization in a single task. In an online experiment, we find that people use structure (when provided) to generalize and perform uncertainty-directed exploration, with structure helping more in larger environments. In environments without structure, exploration becomes more random and more planning is needed. All behavioral effects are captured in a single model with recoverable parameters. In conclusion, our results connect past research on human RL in one framework using correlated trees.

Theories of intrinsic motivation describe how behavior is driven by inherent satisfaction beyond only rewarding outcomes. One computational theory quantifies “fun” as the pleasure derived from improving one’s model of the environment. Here, we refine and test this theory by predicting maximal fun occurs when learning progress is also maximal, corresponding to a balance between ability (or knowledge) and task difficulty. Across multiple natural data sets (e.g., “Super Mario Maker”, “Trackmania”, or “Robozzle”), we confirm our prediction that human judgments of fun are highest at intermediate levels of difficulty. We provide further evidence through a number guessing experiment, where we manipulated the learnability of the environment by controlling the variance of the numbers that could be guessed. Both participants’ engagement and model-based analyses confirmed our predictions. Beyond simply exploiting maximal rewards or exploring maximal uncertainty, the constraints of learnability require a balance between challenge and ease.

How do people decide whether to try out novel options? We argue that they utilize contextual information toefficiently generalize from learned functional relations in order to decide between known or novel options. In a contextualmulti-armed bandit task, in which rewards are a noisy function of observable features, we assess participants’ preferences fornewly introduced options. We show that participants preferably choose a novel option if its features indicate high rewards,but shun the option if its features indicate low rewards, a behavior that can only be explained by functional generalization.Moreover, we assess people’s preferences for novel options that have medium rewards to test whether they prefer options lesssimilar to experienced options, consistent with choices guided by uncertainty. Given that novel options normally come withobservable features, we argue that contextual learning is a parsimonious yet powerful explanation of behavior in the face ofnovelty.

Understanding abstraction is a stepping stone towards understanding intelligence. We ask the question: How do abstract representations arise when learning sequences? From a normative perspective, we show that abstraction is necessary for an intelligent agent when the perceptual sequence contains objects of similar interaction properties appearing in identical contexts. A rational agent should identify categories of objects of similar properties as an abstract concept, enabling the discovery of higher-order sequential relations that span a longer part of the sequence. We propose a hierarchical variable learning model (HVM) that learns chunks and abstract concepts from sequential data in a cognitively plausible manner. HVM gradually discovers abstraction via a conjunction of variable discovery and chunking, resembling the process of concept discovery during development. In a sequence recall experiment that demands learning and transferring variables, we observe that the model's sequence complexity can explain human behavior in a sequence memorization experiment.

How can heuristic strategies emerge from smaller build-ing blocks? We propose Approximate Bayesian Com-putation (ABC) as a computational solution to thisproblem. As a first proof of concept, we demonstratehow a heuristic decision strategy such as Take The Best(TTB) can be learned from smaller, probabilisticallyupdated building blocks. Based on a self-reinforcingsampling scheme, different building blocks are com-bined and, over time, tree-like non-compensatory heuris-tics emerge. This new algorithm, coined ApproximatelyBayesian Computed Take The Best (ABC-TTB), is ableto recover data that was generated by TTB, leads tosensible inferences about cue importance and cue direc-tions, can outperform traditional TTB, and allows totrade-off performance and computational effort explic-itly.

Cognitive search processes are generally affected by the number of available items. We investigated if this also applies to long-term memory retrieval. Specifically, we explored the effect of set size on retrieval times of cued memories from long-term memory. Participants learned lists of word pairs that varied in the number and the semantic similarity of the pairs. An increase in set size resulted in slower retrieval times, indicating the influence of set size on memory retrieval efficiency. However, participants were faster in retrieving more semantically similar word pairs. These findings are consistent with a search-based model of retrieval, illustrating its sensitivity to the number of memory candidates, while highlighting the role of the quality of the cue in optimizing search performance. Furthermore, we established the validity of using similarity values based on Word2Vec embeddings by showing a high correlation with human similarity ratings and similar model results.

Over the last decades, games have become one of the most popular recreational activities, not only among children but also among adults. Consequently, they have also gained popularity as an avenue for studying cognition. Games offer several advantages, such as the possibility to gather big data sets, engage participants to play for a long time, and better resemblance of real world complexities. In this workshop, we will bring together leading researchers from across the cognitive sciences to explore how games can be used to study diverse aspects of intelligent behavior, explore their differences compared to classical lab experiments, and discuss the future of game-based cognitive science research.