Language understanding requires listeners to quickly compresslarge amounts of perceptual information into abstract linguis-tic categories. Critical cues to those categories are distributedacross the speech signal, with some cues appearing substan-tially later. Speech perception would thus be facilitated if gra-dient sub-categorical representations of the input are main-tained in memory, allowing optimal cue integration. How-ever, indiscriminate maintenance of the high-dimensional sig-nal would tax memory systems. We hypothesize that speechperception balances these pressures by maintaining gradientrepresentations that are expected to facilitate category recog-nition. Two perception experiments test this hypothesis. Be-tween participants, an initial exposure phase manipulated theutility of information maintenance: in the High-Informativitygroup, following context always was informative; in the Low-Informativity group, following context always was uninforma-tive. A subsequent test phase measured the extent to whichparticipants maintained gradient representations. The Low-Informativity group showed less maintenance, compared to theHigh-Informativity group (Experiment 1). We then increasedthe task demands and made the targets of the manipulation lessobvious to participants (Experiment 2). We found a qualita-tively similar pattern. Together, these results suggest that lis-teners are capable of allocating memory to gradient representa-tions of the speech input based on the expected utility of thoserepresentations.

Language temporally unfolds, with relevant cues arriving at different moments in time. For comprehension to be optimal,listeners must maintain gradient representations of cues in order to integrate with later-arriving cues. Several studies haveestablished during speech perception listeners integrate cues that occur far apart in time. There are several proposalsabout how restricted this is, but there’s little rigorous work establishing and testing models of long-distance cue integrationstrategies. We take a first step at addressing this gap by formalizing four different models of how listeners use cueinformation during real-time processing, testing them on two perception experiments. In one experiment, we find supportfor optimal integration of cues. In another, more attention-taxing experiment, we find evidence in favor of a strategy thatavoids maintaining detailed representations of cues in memory. These results represent a first step toward understandinghow listeners change their cue integration strategies across contexts.

Exposure to unfamiliar non-native speech tends to improve comprehension. One hypothesis holds that listeners adapt to non-native-accented speech through distributional learning-by inferring the statistics of the talker's phonetic cues. Models based on this hypothesis provide a good fit to incremental changes after exposure to atypical native speech. These models have, however, not previously been applied to non-native accents, which typically differ from native speech in many dimensions. Motivated by a seeming failure to replicate a well-replicated finding from accent adaptation, we use ideal observers to test whether our results can be understood solely based on the statistics of the relevant cue distributions in the native- and non-native-accented speech. The simple computational model we use for this purpose can be used predictively by other researchers working on similar questions. All code and data are shared.

Speech perception is made much harder by variability betweentalkers. As a result, listeners need to adapt to each differenttalker’s particular acoustic cue distributions. Thinking of thisadaptation as a form of statistical inference, we explore the rolethat listeners’ prior expectations play in adapting to an unfa-miliar talker. Specifically, we test the hypothesis that listenerswill have a harder time adapting to talkers whose cue distribu-tions fall outside the range of normal variation across talkers.We also show that it is possible to infer listeners’ shared priorexpectations based on patterns of adaptation to different cuedistributions. This provides a potentially powerful tool for di-rectly probing listeners’ prior expectations about talkers thatdoes not rely on speech produced by many different talkers,which is costly to collect and annotate, and only indirectly re-lated to listeners’ subjective expectations.