Much of cognitive science is based on constructing, validating, and comparing formal models of the mind. Whereascoming up with new and useful models requires expertise and creativity, validating the proposed models and comparingthem against the state-of-the-art mainly requires a systematic, rigorous approach. The task of model validation is thereforeparticularly well-suited for the types of automation that have propelled other research fields (cf. impact of bioinformaticson biology). Here we propose a model benchmarking framework implemented as an open-source Python package namedCogniBench. Given a set of candidate models (which can be implemented in various languages), experimental obser-vations, and scoring criteria, CogniBench automatically performs model benchmarks and reports the resulting matrix ofscores. We demonstrate the potential of the proposed framework by applying it in the domain of learning and decisionmaking, which poses unique requirements for model validation.

Exploration-exploitation of functions, that is learningand optimizing a mapping between inputs and expectedoutputs, is ubiquitous to many real world situations.These situations sometimes require us to avoid certainoutcomes at all cost, for example because they arepoisonous, harmful, or otherwise dangerous. We testparticipants’ behavior in scenarios in which they haveto find the optimum of a function while at the sametime avoid outputs below a certain threshold. Intwo experiments, we find that Safe-Optimization, aGaussian Process-based exploration-exploitation algo-rithm, describes participants’ behavior well and thatparticipants seem to care first about whether a point issafe and then try to pick the optimal point from all suchsafe points. This means that their trade-off betweenexploration and exploitation indicates intelligent,approximate, and homeostasis-driven behavior

Virtual Reality (VR) can offer insights into realistic human defensive behavior. In the present work, we sought to elucidate the interplay between feelings and actions in VR-simulated threatening scenarios. Participants (n = 30) encountered various animal threats in VR during a fruit collection task. We retrospectively assessed participants' feelings after each episode on several dimensions, namely valence, arousal, potency, surprise, and anxiety. As predictor variables, we included scenario characteristics, behavioral responses, and personality traits. Our results indicate that the primary determinants for subjective feelings except potency were ultimate survival, the availability of self-defense weapons, and the animals' behavior (attack or not). No strong determinants for potency could be found. Notably, participants' behavioral responses did not independently influence feelings reported later. These findings highlight VR's potential in expanding our understanding of subjective feelings in threatening situations. Our research suggests that behavior and feelings in defensive situations might not be closely linked.