Proceedings of the Annual Meeting of the Cognitive Science Society

Pragmatic implicature derivation presupposes that the cooperative principle is observed and critically depends on interlocutors expecting each other to behave cooperatively. It is much less clear, however, whether people extend this assumption to communication with artificial agents. People might therefore not draw the same pragmatic inferences when interacting with an artificial agent as they would with other conversationally competent humans, even if the agent is in principle believed to be similarly competent. In our study, we ask participants to interpret messages in a pragmatic reference game which they are told were generated by ChatGPT. Additionally, participants report whether they believe ChatGPT to be capable of the reasoning needed to select the optimal message. We observe a noteworthy discrepancy: in the reference game, participants interpret ChatGPT's messages less pragmatically than those of another adult human, but in the post-test questionnaire, they overwhelmingly rate ChatGPT's pragmatic ability very highly.