Proceedings of the Annual Meeting of the Cognitive Science Society

Listener gaze can predict reference resolution as it reflectslisteners’ understanding. Further, speakers commonly referin installments to co-present objects by providing a descrip-tion incrementally. Here, we investigate whether listener gazecould be utilized to refer incrementally, in spoken installments.Specifically, we implemented a system that generates instruc-tions, describes objects, and reacts to listener gaze with verbalfeedback. We compared unambiguous vs. ambiguous instruc-tions supplemented by two levels of feedback specificity: ei-ther underspecified (“No, not that one!”) or more informative,contrastive responses (“Further left!”). Our findings show thatambiguous instructions with underspecified feedback did notbenefit task performance. In contrast, ambiguous instructionswith contrastive feedback (referring in installments) resulted inmore efficient interactions. Moreover, this strategy even out-performed the one providing unambiguous instructions.