Proceedings of the Annual Meeting of the Cognitive Science Society

We explored (1) the differences in prosody and gesture when speakers were aware and unaware of ambiguities, and (2) the insight of multimodal ambiguity resolution on communication efficiency. Thirty-two Mandarin speakers articulated twenty-two ambiguous Mandarin sentences. Half could be disambiguated using prosody (half couldn't). First, participants articulated each sentence and explained its meaning to a confederate, revealing their dominant interpretation and ambiguity awareness. Second, participants articulated the same ambiguous sentences twice according to hints indicating two meanings. Results showed participants hardly realised ambiguities. Speakers produced mostly more prominent prosody and more gestures when recognising ambiguities. When ambiguity was aware, prosodically unambiguous sentences were produced with various prosodic cues, with referential and non-referential gestures. However, prosodically ambiguous sentences were produced with more referential but hardly any non-referential gestures. In conclusion, speakers adopt multimodal strategies to achieve communication efficiency with a trade-off between modalities, depending on their ambiguity awareness.