Proceedings of the Annual Meeting of the Cognitive Science Society

A question-answering system needs to be able to reason aboutunobserved causes in order to answer questions of the sort thatpeople face in everyday conversations. Recent neural networkmodels that incorporate explicit memory and attention mecha-nisms have taken steps towards this capability. However, thesemodels have not been tested in scenarios for which reasoningabout the unobservable mental states of other agents is nec-essary to answer a question. We propose a new set of tasksinspired by the well-known false-belief test to examine howa recent question-answering model performs in situations thatrequire reasoning about latent mental states. We find that themodel is only successful when the training and test data bearsubstantial similarity, as it memorizes how to answer specificquestions and cannot reason about the causal relationship be-tween actions and latent mental states. We introduce an ex-tension to the model that explicitly simulates the mental rep-resentations of different participants in a reasoning task, andshow that this capacity increases the model’s performance onour theory of mind test.