Proceedings of the Annual Meeting of the Cognitive Science Society

Human–AI collaboration is common, yet AI-assisted decision-making processes among experts in cognitively demanding tasks remain understudied. We investigated how railway planners evaluate advice attributed to either an AI algorithm or a human colleague, and whether witnessing poor advice influences advice acceptance. Seventy-nine railway planners completed an abstract scheduling task. Bayesian hierarchical models showed similar acceptance of AI and human advice, both before and after witnessing poor advice, at the group level. Despite similar average acceptance across groups, models that explicitly capture individual differences indicate heterogeneity only in the AI condition. Some participants discounted the advice after witnessing poor advice, whereas others did not show this discounting. No comparable pattern was evident in the human advice condition. Our results suggest that inconsistent findings in the literature may reflect individual differences in how people respond to algorithmic advice. Future research should therefore focus on identifying and explaining these individual differences.