Proceedings of the Annual Meeting of the Cognitive Science Society

Human communication involves far more than words; speakers’ utterances are often accompanied by various kinds of emotional expressions. How do listeners represent and integrate these distinct sources of information to make communicative inferences? We first show that people, as listeners, integrate both verbal and emotional information when inferring true states of the world and others' communicative goals, and then present computational models that formalize these inferences by considering different ways in which these signals might be generated. Results suggest that while listeners understand that utterances and emotional expressions are generated by a balance of speakers’ informational and social goals, they additionally consider the possibility that emotional expressions are noncommunicative signals that directly reflect the speaker’s internal states. These results are consistent with the predictions of a probabilistic model that integrates goal inferences with linguistic and emotional signals, moving us towards a more complete formal theory of human communicative reasoning.