In current approaches to pragmatic reasoning the comprehension and production of referring expressions is modeled as a result of the interlocutors’ mutual perspective-taking. While such models of pragmatic reasoning have been empirically validated in referential language games experiments, empirical (and computational) work on the generation of referring expressions has shown that speakers do not always take the listener’s perspective into account, but instead produce referring expressions according to their own preferences. One particularly well studied example is color: speakers often include color terms in their referring expressions even if they do not help identify the intended referent. We show that like speakers, listeners treat color differently from other properties like e.g. size. Our results suggest that listeners do not seem to perform much pragmatic reasoning when the referring expression only expresses color, but instead follow a simple saliencebased heuristic.