- Main
Do Models Capture Individuals?Evaluating Parameterized Models for Syllogistic Reasoning
Abstract
The prevailing focus on aggregated data and the lacking group-to-individual generalizability it entails have recently been iden-tified as a major cause for the low performance of cognitivemodels in the field of syllogistic reasoning research. This arti-cle attempts to add to the discussion about the performance ofcurrent syllogistic reasoning models by considering the param-eterization capabilities some cognitive models offer. To thisend, we propose a model evaluation setting targeted specifi-cally toward analyzing the capabilities of a model to fine-tuneits inferential mechanisms to individual human reasoning data.This allows us to (1) quantify the degree to which models areable to capture individual human reasoning behavior, (2) ana-lyze the efficiency of the parameters used by models, and (3)examine the functional differences between the prediction ca-pabilities of competing models on a more detailed level. Weapply this method to two state-of-the-art models for syllogisticreasoning, mReasoner and the Probability Heuristics Model,analyze the obtained results and discuss their implication withrespect to the general field of cognitive modeling.