Words exist for referring at many levels of specificity: from the broadest (thing) to the most specific (Fido). What drives the emergence of these taxonomies of reference? Recent com- putational theories of language evolution suggest that commu- nicative demands of the environment may play a deciding role. Here, we investigate local pragmatic mechanisms of lexical adaptation that may undergird global emergence by manipulat- ing context in a repeated reference game where pairs of partic- ipants interactively coordinate on an artificial communication system. We hypothesize that pairs should converge on specific names (e.g. Fido) when the context requires frequently mak- ing fine distinctions between entities; conversely, they should converge on a more compressed system of conventions for ab- stract categories (e.g. dog) in coarser contexts, even if a finer mapping would be sufficient. We show differences in the lev- els of abstraction that emerged in different environments and introduce a statistical approach to probe the dynamics of emer- gence.

What governs how much information speakers include in referring expressions? Atypical properties of objects aremore likely to be included in referring expressions than typical ones. E.g., speakers are more likely to call a blue banana a “bluebanana” and a yellow banana a ”banana”. A unified account of this phenomenon is lacking. When should a rational speakermention an object’s color? Reference production is modeled within the Rational Speech Act framework. Utterances (“banana”,“blue”, and “blue banana”) are taken to have a graded semantics: rather than assuming all bananas are equally good instancesof “banana”, we empirically elicited object-utterance typicality values for all possible utterances. Pragmatic speakers selectutterances proportionally to the probability that a literal listener using a graded semantics will select the intended referent. Weevaluate the proposed model on a dataset of freely produced referring expressions collected in an interactive reference gameexperiment via the web.

e effortlessly grasp the correspondence between a drawingof an object and that physical object in the world, even whenthe drawing is far from realistic. How are visual objectconcepts organized such that we can both recognize theseabstract correspondences and also flexibly exploit them whencommunicating them to others in a drawing? Here we considerthe notion that the compositional nature of object conceptsenables us to readily decompose both objects and drawings ofobjects into a common set of semantically meaningful parts.To investigate this, we collected data on the part informationexpressed in drawings by having participants densely annotatedrawings of real-world objects. Our dataset contained bothdetailed and sparser drawings produced in different commu-nicative contexts. We found that: (1) people are consistentin what they interpret individual strokes to represent; (2)single strokes tend to correspond to single parts, with strokesrepresenting the same part often being clustered in time; and(3) both sparse and detailed drawings of the same object em-phasize similar part information, although detailed drawingsof different objects are more distinct from one another thansparse drawings. Taken together, our results support the notionthat people flexibly deploy their abstract understanding ofthe compositional part structure of objects to communicaterelevant information about them in context. More broadly,they highlight the importance of structured knowledge forunderstanding how pictorial representations convey meaning.

Nominal reference is very flexible—the same object may becalled a dalmatian, a dog, or an animal when all are literallytrue. What accounts for the choices that speakers make in howthey refer to objects? The addition of modifiers (e.g. big dog)has been extensively explored in the literature, but fewer stud-ies have explored the choice of noun, including its level of ab-straction. We collected freely produced referring expressionsin a multi-player reference game experiment, where we ma-nipulated the object’s context. We find that utterance choiceis affected by the contextual informativeness of a description,its length and frequency, and the typicality of the object forthat description. Finally, we show how these factors naturallyenter into a formal model of production within the RationalSpeech-Acts framework, and that the resulting model predictsour quantitative production data.

What makes a question useful? What makes an answer appropriate? In this paper, we formulate a family of increasingly sophisticated models of question-answer behavior within the Rational Speech Act framework. We compare these models based on three different pieces of evidence: first, we demonstrate how our answerer models capture a classic effect in psycholinguistics showing that an answerer’s level of informativeness varies with the inferred questioner goal, while keeping the question constant. Second, we jointly test the questioner and answerer components of our model based on empirical evidence from a question-answer reasoning game. Third, we examine a special case of this game to further distinguish among the questioner models. We find that sophisticated pragmatic reasoning is needed to account for some of the data. People can use questions to provide cues to the answerer about their interest, and can select answers that are informative about inferred interests.

What cognitive mechanisms support the emergence of linguis-tic conventions from repeated interaction? We present re-sults from a large-scale, multi-player replication of the clas-sic tangrams task, focusing on three foundational propertiesof conventions: arbitrariness, stability, and reduction of ut-terance length over time. These results motivate a theory ofconvention-formation where agents, though initially uncertainabout word meanings in context, assume others are using lan-guage with such knowledge. Thus, agents may learn aboutmeanings by reasoning about a knowledgeable, informativepartner; if all agents engage in such a process, they success-fully coordinate their beliefs, giving rise to a conventionalcommunication system. We formalize this theory in a compu-tational model of language understanding as social inferenceand demonstrate that it produces all three properties in a sim-plified domain.

Drawing is a versatile technique for visual communication,ranging from photorealistic renderings to schematic diagramsconsisting entirely of symbols. How does a medium spanningsuch a broad range of appearances reliably convey meaning? Anatural possibility is that drawings derive meaning from boththeir visual properties as well as shared knowledge betweenpeople who use them to communicate. Here we evaluate thispossibility in a drawing-based reference game in which twoparticipants repeatedly communicated about visual objects.Across a series of controlled experiments, we found that pairsof participants discover increasingly sparse yet effective waysof depicting objects. These gains were specific to thoseobjects that were repeatedly referenced, and went beyond whatcould be explained by task practice or the visual properties ofthe drawings alone. We employed modern techniques fromcomputer vision to characterize how the high-level visual fea-tures of drawings changed, finding that drawings of the sameobject became more consistent within a pair of participants anddivergent across participants from different interactions. Takentogether, these findings suggest that visual communicationpromotes the emergence of depictions whose meanings areincreasingly determined by shared knowledge rather than theirvisual properties alone.

Despite its importance, human collective intelligence remains enigmatic. We know what features are predictive of collective intelligence in human groups, but we do not understand the specific mechanisms that lead to the emergence of this distributed information processing ability. In contrast, there is a well-developed literature of experiments that have exposed the mechanisms of collective intelligence in nonhuman animal species. We adapt a recent experiment designed to study collective sensing in groups of fish in order to better understand the mechanisms that may underly the emergence of collective intelligence in human groups. We find that humans in our experiments act at a high level like fish but with two additional behaviors: independent exploration and targeted copying. These distinctively human activities may partially explain the emergence of collective sensing in our task environment at group sizes and on times scales orders of magnitudes smaller than were observed in fish.